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(54) Device and method for detecting digital watermark information 



(57) The present invention provides a method for 
detecting digital watermark Information Inserted In orig- 
inal image infomnation from image Infomiation in which 
the digital watemriark infomiation is deleted, altered or 
deformed by executing defonning operation to the orig- 
inal image infonmation. The digital watennari< infonma- 
tion detecting device 1 00 comprises: means 2 which cal- 
culates a matrix and a transposed matrix about a system 
matrix from an obtained image Infomnation; means 3 
which calculates an estimated initial image vector from 
the image infonnation; means 4 which calculates a re- 
sidual vector of the image information; means 5 which 
calculates a square en-or of the residual vector; means 
8 which determines whether or not the residual vector 
square error value is a minimum value; means 7 which 
calculates a correcting vector; means 8 which calculates 
a reverse-rotated image; means 9 which substitutes an 
output from the reverse-rotated image calculating 
means with a value calculated by the estimated image 
vector calculating means; means 10 which obtains an 
estimated value of the original image information when 
the minimum value of the residual vector square error 
is detected; and digital watermark infomnation display- 
ing means 11 which retrieves digital watemnark Infomna- 
tion from the estimated Image infonnation of the original 
image infomnation and which displays the retrieved dig- 
ital watemnark infomiation. 
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Description 

BACKGROUND OF THE INVENTION 
5 1 . Field of the Invention 

[0001] The present invention relates to a device and a method for detecting digital watermark infomriation, and more 
specifically, to a device and a method for detecting digital watemiark information, capable of detecting original digital 
watermark Information inserted in original image Infomriation, even when the original image infomnation in which the 

10 digital watemiark Information Inserted Is deleted, defomned or altered by tampering. The present Invention also relates 
to a device and a method for detecting digital watermark infonnatlon, capable of identifying the kind of defomiing 
operation performed from an obtained subject image, reproducing the original image from the obtained image tampered 
by the kind of defomiing operation, and detecting the digital watemnark information originally inserted In the original 
image infomnation even when the original image Infomriation in which the digital watermark Infonnation is inserted has 

15 been deleted, defomned or altered by tampering. 

2. Description of the Related Art 

[0002] The still picture digital watemnark technology has been utilized for secretly embedding infonnation unique to 

20 an author In a still picture, as a technology to provide basis for claiming proprietary right. 

[0003] In recent years, wfth an increasing amount of distribution of multimedia infomnation through the use of Internet, 
application of the digital watermark technology has been drawing attention. At the same time, however, malicious 
conduct is on increase, in which watemiark information embedded in various kinds of images is defomned, altered or 
deleted for the purpose of avoiding proprietary duty, and therefore development of a method for preventing these kinds 

25 of Illegal act is an urgent requirement. 

[0004] In the Increase of the tampering in which the digital watermark information is deformed, altered or deleted, it 
is commonly known that the digital watemnark Infomnation can be easily deleted, defomned or altered by purposefully 
executing such an operation as enlarging or reducing, and rotating the original image infomnation. 
[0005] For example, the digital watemnark infonnation can be easily deleted by perfomning the rotating deforming 

30 operation to the original image infonnation. 

[0006] Specifically, if the Image in which the digital watermark infomnation is embedded is defomned by rotation at a 
certain angle, then it becomes impossible to accurately detect the digital watemnark Information. 
[0007] That is, when such a defomiing operation as described above is perfomned to original image infonnation, the 
image infonnation itself will not be entirely lost by the defomnlng operation even if quality of the image may be slightly 

35 lowered, since the image information often comprises identical infonnation shared by mutually adjacent pixels. How- 
ever, the digital watemnark infonnation, which consist of a randomly laid out binary values of 0 and 1 , can be deleted 
by an averaging operation utilized In the above defomnlng operation. For this reason, the original digital watemnark 
infonnation can often become illegible, or altered into digital watemnark infomnation which differs from the original digital 
watennari< infonnation. 

40 [0008] Because those who intend to do illegal act can delete the digital watermark infonnation without hopelessly 
deteriorating the quality of the image itself, such means is now practiced very often. 

[0009] At present, commonly employed methods for wrongdoing (attack) the digital watermark infomnation include 
a rotating defomnlng operation and an enlarging/reducing defomnlng operation for example, performed to information 
processed by an affine transfomnation method. 
45 [0010] Further, methods commonly employed to the rotating deforming operation and the enlarging/reducing de- 
fomiing operation includes a linear interpolation method and a nearest-neighborhood method by means of an affine 
transfomnation method. 

[0011] Specifically, in such a tampering (attack), the rotating deforming operation and enlarging/reducing defomning 
operation executed to the original image will not cause a great damage to the original Image itself, yet Intensity of the 

50 embedded digital watermark infonnation will be decreased, sometime to an undetectable degree. 

[0012] Since there has been no effective and reliable preventing method against such tampering, development of 
the preventing method has been an urgent requirement. Another example of the operation by which the digital water- 
mark is easily deleted or made Illegible is a defomning operation by enlarging/reducing the image. 
[001 3] Specifically, if the size of an original image In which the digital watemnark Infomnation is embedded is changed 

S5 from an image size of the original image by enlarging/reducing the image, the digital watermark Infonnation can be 
deleted, or shifted from a location of original embedment due to addition of black infonnation. Thus, it becomes im- 
possible to accurately detect the digital watemnaric infonnation once the image is enlarged or reduced. 
[001 4] Particulariy, the digital watermaric information has a characteristic that can be easily deleted by the defomnation 
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caused by the enlarging/reducing deforming operation. 

[0015] In other words, if the image in which digital watennark infonnation is embedded is enlarged/reduced by using 
a certain enlarging/reducin g rate, then the digital watermark information becomes unable to be detected con-ectly. 
[0016] More specifically, if such a defomiing operation as described above is perfomned to the original image Infor- 

5 mation, the original image information itself will not be totally lost even if quality of the image may be slightly lowered, 
since the image information often comprises identical information shared by mutually adjacent pixels. However, the 
digital watermark infonnation, which consists of randomly laid out binary values of 0 and 1, can be deleted by an 
averaging operation utilized in the above deforming operation. Forth is reason, the original digital watennark infonnation 
can often become illegible, or even if legible, become altered into digital watennark infonnation which differs from the 

10 original digital watennark information. 

[0017] Because those who intend to do wrong act can delete the digital watennark Information without hopelessly 
deteriorating the quality of the image itself, such means is now practiced very often. 

[0018] At present, a commonly employed method to counter such a tampering include a linear interpolation method 
by means of an affine transfomnation method, for bringing the enlarged or reduced image back into the original size, 
IS or bringing the rotated image back to the original image by using an estimated angle of rotation. 

[0019] However, the linear interpolation method by means of an affine transformation can decrease intensity of the 
embedded watennark to an undetectable degree. 

[uuzuj l-urther, when the original Image is not obtained, the size of the original image is not available, and it is 
therefore difficult to identify the size of the original image from the obtained image which is already enlarged/reduced. 
20 [0021] Thus, since there has been conventionally no effective and reliable defending method against such tampering, 
development of the method has been an urgent requirement. 

SUMMARY OF THE INVENTION 

25 [0022] It Is therefore an object of at least the preferred embodiments of the present invention to improve the above 
described shortcomings in the prior art, thereby to provide a method and a device for detecting digital watenrnaric 
infonnation, capable of reliably detecting an original digital watennaric infonnation embedded In original image Infor- 
mation, from image infonnation, particularly from image information made by rotating the original image thereby having 
the original digital watennark infonnation deleted, altered or defonned. 

30 [0023] Further, It is another such object to provide a device and a method for detecting digital watennark Infonnation 
necessary for protecting proprietary right to the Image information from Illegal infringement, by making possible to 
detect the original digital watermaric infonnation which is still image infonnation distributed in the maricet prepared 
illegally after executing the rotating defomiing operation to the original still image infonnation which contains the pre* 
detennined digital watennark information. 

35 [0024] Further, it Is another such object to improve the above described shortcomings in the prior art, thereby to 
provide a method and a device for detennining whether or not defonned image information is formed by enlarging/ 
reducing an original image from the defonned image information In which the digital watennark Information has been 
deleted from the image after the defonning operation, or in which it has become illegible, as a result of enlarging or 
reducing deforming operation to original image infonnation containing digital watennark infonnation, and to provide a 

40 method and a device for detecting an image size of an original Image by identifying whether or not the defomned image 
infonnation is enlarged/reduced by using to what degree enlarging/reducing rate to the original image. At the same 
time, it is to provide a method for detecting digital watennaric infonnation, capable of reliably detecting original digital 
watennark infonnation embedded in original image infonnation, from the deformed image infonnation. 
[0025] Further, another such object is to provide methods for detecting a size of an original image and digital water- 

45 mart<, necessary for protecting proprietary right to the image infonnation from illegal infringement, by making possible 
to detectthe original digital watennark infomnation from still image information distributed in the maricet prepared illegally 
after executing enlarging or reducing operation by a third person to the original still image information which contains 
the predetennined digital watennark Infonnation and whose origin is definite. 

[0026] A first aspect of the present invention provides a digital watermaric information detecting device comprising: 
so analyzing means which analyzes an obtained subject image information expected to contain obtained digital watermark 
infonnation, therein; image processing infonnation detennining means which determines whether the subject image 
infonnation is image infonnation subjected to a rotating defonning operation or image infonnation subjected to an 
enlarging/reducing defonning operation; characteristic value calculating means which calculates a value of a prede- 
tennined characteristic to the subject image infonnation based on a result of the above analysis and determination; 
S5 image infonnation estimating means which estimates image information of the subject image infonnation by analyzing 
the characteristic value; and digital watennaric information retrieving means which retrieves digital watennaric infonna- 
tion contained in the estimated image infonnation. 

[0027] A second aspect of the present Invention provides a digital watennaric infonnation detecting device comprising: 
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image information analyzing means for an obtained image; matrix calculating means which calculates a matrix and a 
transposed matrix about a system matrix, from the image information; estimated initial image vector calculating means 
which calculates an estimated Initial image vectorf rom the image information; residual vector calculating means which 
calculates a residual vector between the image vector in the image information and a value obtained by multiplying 
5 the estimated Initial image vector with the matrix; .residual vector square error calculating means which calculates a 
square error of the residual vector; square error value determining means which detemnines whether or not the residual 
vector square error value is a minimum value; correcting vector calculating means which calculates a correcting vector; 
reverse-rotated image calculating means which calculates a reverse-rotated image; adding means which adds an 
output from the reverse-rotated image calculating means to a value calculated by the estimated image vector calculating 
10 means; means which obtains an estimated value of the original image infomriation from the minimum value of the 
> residual vector square error when the square error value detennining means detects the minimum value of the residual 

vector square en-or; digital watemiark infomriation displaying means which retrieves digital watermark infomriation from 
the estimated image information of the original Image infomriation and which displays the retrieved digital watemriark 
Infonnation. 

IS [0028] Next, a third aspect of the present invention provides a digital watemriark infonnation detecting method com- 
prising the steps of: obtaining a subject image information expected to contain digital watenmark infonnation; perfomning 
analysis on the subject image infomriation, thereby analyzing and detennining whether the Ruhiftf^t imarje Information 
is image information subjected to a rotating deforming operation or image information subjected to an enlarging/re- 
ducing defonning operation; obtaining a value of a predetemnined characteristic to the subject image infomriation based 

20 on a result of the above analysis and determination; estimating image information of the subject image infonnation by 
analyzing the characteristic value; and retrieving digital watermark information contained in the estimated Image infor- 
mation. 

[0029] Further, a fourth aspect of the present invention provides a digital watennark information detecting method 
comprising the steps of: obtaining image infonnation; determining whether or not the obtained image infonnation was 

25 subjected to a rotating deforming operation; calculating a matrix and a transposed matrix about a system matrix from 
the image infonnation deformed by rotation; calculating an estimated initial image vector from the image infonnation; 
calculating a residual vector between the Image vector in the image infonnation and a value obtained by multiplying 
the estimated initial image vector with the matrix; calculating a square error of the residual vector; detennining whether 
or not the residual vector square error value is a minimum value; calculating a correcting vector and calculating a 

30 reverse-rotated image when the square en-or value of the residual vector is determined not to be the minimum value; 
adding the calculated reverse-rotated image infonnation to a value calculated by the estimated Image vector calculating 
means; repeating the residual vector square en^or calculation as necessary; estimating original image infomriation from 
the minimum square error value when the square error value of the residual vector is determined to be the minimum 
value; and retrieving digital watennark infonnation from the estimated original image information. 

35 [0030] Further, a fifth aspect of the present invention provides a recording medium storing a program for executing 
a digital watennark information detecting method by computer, the method comprising: a first step of obtaining image 
infonnation; a second step of detennining whether or not the obtained Image infonnation was subjected to a rotating 
defonning operation; a third step of calculating a matrix and a transposed matrix about a system matrix from the image 
infomriation defomned by rotation; a fourth step of calculating an estimated initial image vector from the image infor- 

40 mation; a fifth step of calculating a residual vector between the image vector in the image infonnation and a value 
obtained by multiplying the estimated Initial image vector with the matrix; a sixth step of calculating a square error of 
the residual vector; a seventh step of detennining whether or not the residual vector square error value is a minimum 
value; an eighth step of calculating a correcting vector when the square error value of the residual vector is determined 
not to be the minimum value; a ninth step of calculating a reverse-rotated image from the correcting vector; a tenth 

45 step of adding the reverse-rotated image infonnation thus obtained to a value calculated by the estimated image vector 
calculating means; an eleventh step of repeating the residual vector square error calculation as necessary; a twelfth 
step of estimating original image infonnation from the minimum square error value when the square en-or value of the 
residual vector is detemnined to be the minimum value; and a step of retrieving digital watemnaric information from the 
estimated original Image Infonnation. 

50 [0031] On the other hand, a sixth aspect of the present invention provides an original image deforming-method 
detecting method, comprising the steps of: embedding a predetermined reference information signal to a predetemnined 
location of a Fourier-space image prepared by Fourier transfomriing an obtained subject image; reverse- Fourier-trans- 
fomning the Fourier-space Image into an image-space image; individually changing the image size of the image-space 
Image by using each one of a plurality of enlarging/reducing rates selected from a predetermined range of the enlarging/ 

55 reducing rates; Fourier transforming again each of the differently-sized image-space images into Fourier-space images; 
detecting frequency components of the embedded reference infonnation signal previously embedded into the image 
information from each of the Fourier-space images; arranging the frequency components of a plurality of the reference 
infonnation signal detected from the a second Fourier-space images of a plurality of the image-space images each 
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having a different size from the other, sequentially according to the size of the enlarging/reducing rates; and detenmining 
that the subject image was made by enlarging/reducing a prescribed original image when it can be detennined that 
there is a frequency component value at a certain enlarging/reducing rate, outstanding from surrounding frequency 
component values. 

5 [0032] A seventh aspect of the present invention provides an original image size detecting method comprising the 
steps of: embedding a predetermined reference infomriation signal to a predetennined location of a Fourier-space 
image prepared by Fourier transfomiing an obtained image; reverse-Fourier-transforming the Fourier-space image 
into an image-space image; individually changing the image size of the image-space image using each one of a plurality 
of enlarging/reducing rates selected from a predetermined range of the enlarging/reducing rates; Fourier transfomning 

10 again each of the differently-sized image-space images into Fourier-space Images; detecting frequency component 
values of the embedded reference infonnation signal previously embedded Into the image to be Inspected, from each 
of the Fourier-space images; arranging the frequency components of said reference information signal detected from 
the Fourier-space images of the image-space images each having a different size from the other, sequentially according 
to the size of the enlarging/reducing rates; and detemiining, when there is a frequency component value at a certain 

15 enlarging/reducing rate outstanding from surrounding frequency component values, that the obtained image was made 
by enlarging/reduci ng an obtained original image at the enlarging/reducing rate corresponding to a frequency com- 
ponent value Wnlch is Outstanding from oijrrcundlng froq'jency component value. 

[0033] Further, an eighth aspect of the present invention provides a digital-watemnark detecting method an-anged to 
perform the steps of: reproducing an original image by applying to an obtained subject image a reciprocal number of 
20 the enlarging/reducing rate at which the size of the original Image was altered to that of the subject Image, the rate 
being obtained by the second aspect of the present invention; and detecting the reproduced original image for a wa- 
termark. 

[0034] A ninth aspect of the present invention provides an original-image-size detecting device, for detecting the 
size of the original image from an obtained image, the device comprising: first Fourier transfonnlng means which Fourier 
25 transfomis the obtained Image Into a first Fourier-space image; reference information signal embedding means which 
embeds a predetennined reference infonnation signal at a predetermined location of the Fourier-space image prepared 
by Fourier transforming the obtained image; reverse Fourier transfomning means which reverse-Fourier-transfomns 
the Fourier-space image into which the reference Infonnation signal had been embedded therein, into an image-space 
image ; enlarging/reducing rate storing means which stores a plurality of enlarging/reducing rates for changing the size 
30 of the image-space image; Image-size changing means which sequentially selects desired enlarging/reducing rates 
from the enlarging/reducing rate storing means, and sequentially and individually changes the image size of the image- 
space image by individually using each of a plurality of the selected enlarging/reducing rates; second Fourier trans- 
fomning means which Fourier again transfomis each of the image-space images the Image size of which being changed, 
respectively, into second Fourier-space images; frequency component value detecting means which detects a frequen- 
ts cy component value of the reference infonnation signal from each of the second Fourier-space images; frequency 
component value arranging means which arrays a plurality of frequency component values obtained from each of the 
second Fourier-space images into a predetennined sequence according to the enlarging/reducing rates; outstanding 
frequency component value detennining means which determines whether or not there is a frequency component value 
outstanding from surrounding frequency component values; and original-image size identifying means which deter- 
40 mines, when the outstanding frequency component is found, that the enlarging/reducing rate corresponding to the 
outstanding frequency component value is the enlarging/reducing rate whereby the image size of said obtained image 
had been changed from that of said original image. 

[0035] On the other hand, a tenth aspect of the present invention provides a recording medium storing a program 
for executing an original image size detecting method, the method comprising: 

45 

a first step of obtaining a subject image which looks similar to an original image from a known source but whose 
digital watemnark information is not legible; 

a second step of Fourier transfomning the obtained subject image Into a first Fourier-space image; 
a third step of embedding a predetermined reference infonnation signal to a predetermined location of the first 
50 Fourier-space image; 

a fourth step of reverse-Fourier-transforming the Fourier-space image provided with the reference infonnation 
signal into an image-space Image; 

a fifth step of Individually changing the size of the image-space image using each one of a plurality of enlarging/ 
reducing rates selected from a predetermined range of the enlarging/reducing rates; 
55 a six step of Fourier transfomning again each of the differently-sized image-space images Into second Fourier- 
space images; 

a seventh step of detecting a frequency component value of the embedded reference infonnation signal previously 
embedded into said Fourier-space images, from each of the Fourier-space images; 
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an eighth step of arranging each of the detected frequency components of said reference information signal, se- 
quentially according to the size of the enlarging/reducing rates; and 

a ninth step of determining whether or not there is a frequency component value outstanding from surrounding 
frequency component values at a certain enlarging/reducing rate among said sequentially arranged frequency 
5 component values, and detemnining, when there is the frequency component value outstanding from surrounding 

frequency component values, that the obtained subject image was made from said original image by changing 
said image size with applying the enlarging/reducing rate con-esponding to said outstanding frequency component 
value, to said original Image. 

10 [0036] Further an eleventh aspect of the present invention provides a recording medium storing a program for exe- 
cuting an original image dig'rtal watemnark detecting method, the method comprising: 

a first step of obtaining a subject image which looks similar to an original Image from a known source but whose 
digital watermark Information is not legible; 
15 a second step of Fourier transfonning the subject image into a first Fourier-space Image; 

a third step of embedding a predetemnined reference infonnatlon signal to a predetermined location of the first 

Fourier-space image; 

a fourth step of reverse- Fourier-transforming the Fourier-space Image provided with the reference information 
signal into an image-space image; 
20 a fifth step of individually changing the size of the image-space image using each one of a plurality of enlarging/ 
reducing rates selected from a predetermined range of the enlarging/reducing rates; 

a six step of Fourier transfonning again each of the differently-sized image-space images into second Fourier- 
space images; 

a seventh step of detecting a frequency component value of the embedded reference information signal previously 
25 embedded Into said Fourier-space images, from each of the Fourier-space images; 

an eighth step of arranging each of the detected frequency components of said reference infonnatlon signal, se- 
quentially according to the size of the enlarging/reducing rates; and 

a ninth step of detemnining whether or not there is a frequency component value outstanding from surrounding 
frequency component values at a certain enlarging/reducing rate among said sequentially arranged frequency 
30 component values, and detemnining, when there is the frequency component value outstanding from surrounding 

frequency component values, that the obtained subject image was made from said original image by changing 
said image size with applying the enlarging/reducing rate con-esponding to said outstanding frequency component 
value, to said original image. 

a tenth step of reproducing an original image by applying a reciprocal number of the enlarging/reducing rate ob- 
35 tained in the ninth step; and 

an eleventh step of determining the original image for watemnark infonnation. 

[0037] Since the digital watermari< infonnation detecting device and the digital watemnark information detecting meth- 
od according to the present invention utilizes one or more of the technical configurations described above, the present 

40 invention can make possible to easily detect an original digital watermark information from an illegally prepared still 
image infonnation being distributed in the mari<et, which was created by a third person through a rotating defomning 
operation and so on executed to original image infonnation which Is a still image containing a predetemnined digital 
watermark infonnation. The present invention thereby can make possible to detemnine whether or not the image infor- 
mation being distributed in the mari<et is made by illegally altering, deforming or deleting a predetemnined digital wa- 

45 temnark information contained in the original image information for a purpose of illegal sales without consent by the 
proprietor. Thus, the present invention can provide a useful means for enabling the proprietor to detect illegal infringe- 
ment. 

[0038] On the other hand, the original image defomning-method detecting method, the original image size detecting 
method and the digital-watennark detecting method according to the present invention utilizes one or more of the 

so technical configurations described above. Therefore, the present invention can make it possible to easily detect what 
kind of defomning operation was performed to the original image, from illegally prepared still Image infonnation being 
distributed in the market, which was created by a third person through an enlarging operation or a reducing operation 
executed to the original image infonnation which is a still image containing a predetennined digital watemnark infor- 
mation. The present invention-thereby can make possible to detemnlne whether or not the image infonnation being 

55 distributed in the market is made by illegally deleting, or making illegible, a predetennined digital watemnark information 
contained in the original image infonnation for a purpose of illegal sales without consent of the proprietor. Thus, the 
present invention can provide a useful means for enabling the proprietor to detect illegal infringement. 
[0039] Preferred features of the invention are as set out in the dependent claims, the substance of which are deemed 
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repeated here as consistory clauses. 
BRIEF DESCRIPTION OF THE DRAWINGS 
5 [0040] 

Fig. 1 rs a block diagram showing a constitution of an embodiment of a digital watennark infomnation detecting 
device according to the present invention; 

Fig. 2 is a diagram showing examples of a method for deforming image infomnation; 
10 Fig. 3(A) and 3(B) are diagrams Illustrating an example of linear interpolation method used when defomning the 

image infonnation; 

Fig. 4 is a diagram illustrating how image infonnation is altered when the image infomnation is deformed by rotation; 
Fig. 5 is a flowchart showing steps of operations in an embodiment of a digital watennark infonnation detecting 
method according to the present invention; 
15 Fig. 6 is a flowchart showing steps of operations in another embodiment of the digital watennark Infonnation de- 

tecting method according to the present invention; 

Fiy . 7 Is a ciiagi am mustratir.g ar, axarr;p!c of a method according to the present Invention, ^or Hetftrmining whether 
or not a still image is defomned by rotation; 

Fig. 8 is a diagram illustrating a calculation procedure in a method for obtaining an estimated image value according 
20 to the present invention; 

Fig. 9 Is a diagram illustrating a calculation procedure in the method for obtaining an estimated image value ac- 
cording to the present Invention; 

Fig. 10 is a diagram illustrating a calculation procedure in the method for obtaining an estimated image value 
according to the present invention; 
25 Fig. 11 a diagram illustrating a calculation procedure in the method for obtaining an estimated Image value ac- 

cording to the present invention; 

Fig. 12 a diagram illustrating a calculation procedure In the method for obtaining an estimated image value ac- 
cording to the present invention; 

Fig. 13 a diagram illustrating a calculation procedure in the method for obtaining an estimated image value ac- 
30 cording to the present Invention; 

Fig. 14 is a graph showing a calculation result in the method for obtaining an estimated image value according to 
the present invention; 

Fig. 1 5 is a graph showing a calculation result in the method for obtaining an estimated image value according to 
the present invention; 

35 Fig. 1 6 Is a block diagram showing a constitution of an embodiment of a device for detecting a defonning method 

applied to an original image, a device for detecting the size of the original image, and a device for detecting a 
digital watermark, according to the present invention; 

Fig. 17 is a diagram showing a state of a signal frequency of an embedded predetemilned signal when an image 
having the signal embedded in an image-space image is Fourier transformed Into a Fourier-space image; 
40 Fig. 18 is a diagram showing a state of a signal frequency of an embedded predetennined signal when an image 

having the signal embedded in an Fourier-space image is reverse-Fourier-transfomned into an image-space image; 
Fig. 19(A) through Fig. 19(C) are diagrams showing a location of a frequency when a signal cut out from a one- 
dimensional signal without changing the size of image is Fourier transformed; 

Fig. 20(A) through Fig. 20(C) are diagrams showing a location of a frequency when a signal cut out from a one- 
45 dimensional signal after reducing the size of image is Fourier transfonned; 

Fig. 21 (A) through Fig. 21 (C) are diagrams showing a location of a frequency when a signal cut out from a one- 
dimensional signal after enlarging the size of image is Fourier transformed; 

Fig. 22 is a diagram showing the locations of the frequency obtained in Fig. 19 through Fig. 21 are shown in one 

diagram; 

50 Fig. 23(A) and Fig. 23(B) are diagrams showing how location of embedment of an embedded signal shifts when 

a two-dimensional signal, i.e. an image containing the embedded signal, is enlarged/reduced and then Fourier 
transformed; 

Fig. 24 is a graph showing an example of relationship between the change of enlargement/reduction rate and an 
average value of frequency components of the location of embedment of the embedded signal, when the Image 
55 containing the embedded signal is enlarged/reduced and then Fourier transfomried; 

Fig. 25 is a flowchart showing an example of an operation procedure of a method for detecting a deforming method 
applied to an original image, a method for detecting the size of the original image, and a method for detecting a 
digital watermark, according to the present Invention; 
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Fig. 26 is a diagram illustrating such steps as embedding of a reference information signal, Fourier transfonning, 
and cutting of a block image after the enlarging/reducing deforming operation in a method for detecting a deforming 
method applied to an original image, a method for detecting the size of the original image, and a method for 
detecting a digital watemnark, according to the present Invention; 

Fig. 27 Is a diagram showing how a location of the reference information signal embedded into the original image 
is shifted by the enlarging/reducing deforming operation; 

Fig. 28 is a diagram showing the enlarging/reducing deforming operation being perfomned to an obtained original 
image; 

Fig. 29 is a graph showing average values of frequency components detected when a subject image Is not enlarged 
or reduced; 

Fig. 30 is a graph showing average values of the frequency components detected when an obtained Image is 
reduced by the rate 1 .2; and 

Fig. 3 1 is a graph showing average values of the frequency components detected when a subject image is enlarged 
by the rate 0.8. 

15 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0041] Hereinafter, embodiments of a device and a method for detecting digital watermark information according to 
the present invention will be described in detail with reference to the accompanying drawings. 

20 [0042] Specifically, Fig. 1 Is a block diagram showing a configuration of a digital watennark infomnation detecting 
device 100 according to the present invention. In this figure, the digital watennark infomiation detecting device 100 
comprises: image infomnation analyzing means 1 for an obtained subject image; matrix calculating means 2 which 
calculates a matrix and a transposed matrix about a system matrix, from the image information; estimated initial image 
vector calculating means 3 which calculates an estimated initial image vector from the image infomnation; residual 

25 vector calculating means 4 which calculates a residual vector between the image vector in the image information and 
a value obtained by multiplying the estimated initial image vector with the matrix; residual vectorsquare en^or calculating 
means 5 which cateulates a square en^orof the residual vector; square error value detemiining means 6 which deter- 
mines whether or not the residual vectorsquare error value of the residual vector is a minimum value; correcting vector 
calculating means 7 which calculates a correcting vector; reverse- rotated image calculating means 8 which calculates 

30 a reverse- rotated image; adding means 9 which adds an output from the reverse-rotated image calculating means to 
a value calculated by the estimated image vector calculating means; means 10 which obtains an estimated value of 
the original image Infomiation from the minimum value of the residual vector square error when the square error value 
detennining means 6 detects the minimum value of the residual vector square eror; digital watemnark information 
displaying means 11 which retrieves digital watennark infonnation from the estimated image infomnation of the original 

35 Image infonnation and which displays the retrieved digital watemnark information; and central calculating means 12 
which integrally controls each of the means listed above. 

[0043] Image infomnation as a subject of analysis in the digital watermark information detecting device according to 

the present invention is still image information. 

[0044] First, description will cover an example showing how digital watennaric infonnation, which is the subject of 
40 the digital watemiark infomnation detecting method according to the present invention, can be deleted or defomned, 
with reference to Fig. 2. 

[0045] Specificaily, methods generally known for altering, defonning or deleting the digital watemnari< infonnation 

embedded Into the original image information include, as described eariier, execution of a defonning operation to the 
original image information. One of the known methods is two-dimensional affine transfomnation method as shown in 
45 Fig. 2. 

[0046] The two-dimensional affine transfomnation method includes a translating type shown in Fig, 2(a), an enlarging/ 
reducing type as shown in Fig. 2(b), and a rotating type as shown in Fig. 2(c). For the purpose of deleting, defomning 
or altering the digital watemnark infonnation, the two-dimensional affine transfomnation method of the enlarging/reduc- 
ing type shown in Fig. 2(b) and the rotating type shown in Fig. 2(c) are primarily employed. 
50 [0047] In the two-dimensional affine transfomnation, in which a two-dimensional image can be freely enlarged, re- 
duced and rotated as shown in Fig. 2(a) and Fig. 2(b), coordinate values of each pixel can be expressed by the following 
Expression 1 , with an optional point (xO, yO) being the center: 



X = ((-^r -x^)costf -(r - y,)sin B)/a + 
y = ((JT - Xo)sin ^ + (y - y^)cos0)/b + 
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wherein X, Y and x, y respectively represent coordinate values of the location of each pixel before and after the affine 
transformation. 

[0048] In the present invention, as has been described earlier, it is necessary to establish a method capable of 
countering the deletion or defonnation of the digital watennark infomiatlon performed by the rotating deforming oper- 
5 ation which utilizes the rotating-type two-dimensional affinetransfomriation method as shown in fig. 2(c). Forthis reason, 
description hereinafter will only cover the rotating deforming operation performed to the image infonnation. 
[0049] Specifically, when the image having the origin (0, 0) is rotated by an angles according to the affine transfor- 
mation given by the Expression 1 , location of each pixel after the rotation will be given by the following Expression 2: 

10 

Expression 2 x^Xcoze^YsmG 

15 

[0050] However, as shown in Fig. 3(A). the address (x, y) given by the Expressions 1 and 2 does not always coincide 
with the pixel (lattice point) or ine inpui image. 

[0051] In such a case as above, a method called the linear interpolation method is perfomned for improved quality 
of the image. 

20 [0052] In the liner interpolation method, as shown in Fig. 3(A), If the given address is not equal to the lattice point, 
distance ratios from adjacent four lattice points are obtained, and the correction Is made by using thickness values of 
these four pixels in proportion to the obtained ratios. Calculation for the thickness value is represented by the following 
Expression 3: 

23 ^ 

d (r, j;) = (I ^ q){{\ .p) + p • d{[x] + !,[>])} 

Expression 3 ^ ? {d - /^) • ^(W, W 1) + • d{[x] + 1, b] + D} 

30 wherein pC] and [Y] are Integers not greater than x and y respectively. 

[0053] Another known method of correction is nearest-neighborhood method. 

[0054] In the nearest-neighborhood method, as shown in Fig. 3(B), the thickness at the nearest point Is selected if 
the location of the pixel given by the Expressions (1) and (2) Is not represent the lattice point of the Input image. 
[0055] In the nearest-neighborhood method, an unintended pixel which happens to be the next pixel can be picked 
35 up. For example, data representing a single line (single row) in the original image can be mistakenly connected by data 
representing the next line (next row) after a rotation. 

[0056] In the linear interpolation method, although image quality is better than is obtained by the nearest-neighbor- 
hood method(in which the thickness of the closest point is selected), intensity of the embedded watermark infomnation 
can be decreased or averaged. 
40 [0057] When the original image information is rotated, these phenomena will become the causes for the deletion, 
deformation or alteration of the digital watemnark infonnation inserted in the original image information, as has been 
described earlier. 

[0058] If the rotated image is rotated again in the same direction, or further rotated in the other direction back into 
the previous angle, detection of the digital watermark infonnation becomes more difficult. 
45 [0059] Now, the problem of rotation and reverse rotation of the image as has been described above is, as shown in 
Fig. 4, a problem of a linear system. 

[0060] H and Mt shown in Fig. 4 are respectively called the system matrix and the transposed matrix of the matrix. 
[0081] If the size of the original image and the rotated image is represented by m x n = k, then the system matrix H 
is expressed as a matrix of k x k. 
50 [0062] From this relationship, the image g' made by rotating the original image information g by a given rotating angle 
is expressed by the following Expression 4: 

Expression 4 g' = Hg 

55 

[0063] And, the Expression 4 can be transfonned into the following Expression 5, through substitution by the Ex- 
pression 2 and the Expression 3: 



9 



EP1 156 660 A2 



Expression 5 
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wherein h(a, b), (0 g a < k), (0 ^ b < k) are elements of H, and their values are determined by g'(i', • n+j =a), 
g(i J)( i • n4-j'=b) and the Expression (3). For example, the pixel g'(0, 0) of the rotated image can be expressed in the 
2s following Expression 6 if corresponding to the pixel g(i, j): 

Expressions g'(O.O) - f ([/].(^-]) 

= (1 - - p) ' gimn) + p ■ gm + 1. m)} 
+ ?{(! - p) ■ gmAn +!)+/'• g{u\ + + 1)} 

35 [0064] From the Expression 6, the elements of the first row of the H respectively take values given by the following 
Expression 7: 

Expression 7 V/.m+i) =(1-^)0-P): V/m+M) = (1-<7)P; 

V0+1)m+i)=^(1"P); 
V(y+1)m+/+1) ^P' 
45 V^) = °' 

[/f(a,fc)9ft(Oj^m+0,(O,/.m+/+1),(O,/+1)m+0,(O,(/+1)n7+/+1)] 

[0065] Next, consideration will be made to the problem that image information deformed by rotating original image 
50 at a certain angle is brought back to the original state by an inverse rotation. The problem can be understood as an 
inverse linear problem. 

[0066] Specifically, the original image, i.e. the original image infonnation g, can be estimated by using the system 
matrix H, from the image Information g' which is the information after the rotation by the given angle. 
[0067] This indicates that if image infonnation Is available and this image infomaation is an image made by executing 
55 the above described rotating deforming operation to original image infonnation which has a certain digital watennark 
information embedded, thereby having the digital watermark infonnation altered or deleted, then the original image 
infonnation can be estimated and at the same time the digital watermark infomiation originally embedded can be 
retrieved by analyzing and finding solutions of the inverse linear problem. 
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[0068] Now, in estimating the original innage information g from the image information g' which is believed to be made 
by rotating the original Image, let the estimated original image infonnation be represented by g". As a scale to measure 
probability of the estimation of the estimated image g", a square error value I represented by the Expression 8 can be 
used, which is the square error value between the product Hg" of the system matrix H and the estimated image g", 
5 and the illegally rotated image infonnation g'. 

Expression 8 I = Ig' - Hg"|^ 

io [0069] Therefore, solving the inverse linear problem means to estimate the estimated original image infonnation g" 
which minimizes the square error I In the above Expression 8. 

[0070] Generally, an affine inverse matrix is used as means for finding solutions of the least square error method in 
the Expression 8. The estimated image infomiation g" is given by the following Expression 9: 



IS 



20 



45 



55 



Expression 9: 



= (H^H)"»H^g' 



[0071] Where, H+ := (HT • H)HT is called the system Inverse matrix. 

[0072] This method of using the affine inverse matrix is advantageous in that the answer can be obtained from a 
simple multiplication of the matrix. However, when the Image to be processed Is large, it is very difficult to obtain this 
25 inverse matrix. Therefore, in actual image processing, a numerical solution to be described below is often used. 

[0073] Therefore, the numerical solution as another method forf inding the solution from the least square error method 
represented by the Expression 8, that is, the estimated image infonnation vector g" of the original image infonnation 
Includes method of steepest descent, conjugate gradient method, and so on. Among other however, it is desirable to 
use a Jacobian method. 

30 [0074] For this reason, hereinafter, description will be made for another technique which uses the Jacobian method 
for obtaining solution to the least square en^or method in the Expression 8. 
[0075] The Jacobian method has the following basic algorithm: 

(1) Give an optional initial estimated Image infomiation vector g"0. 
35 (2) Using the given optional initial estimated Image infonnation vector g''0, find a rotated image vector Hg^O of the 
image from the Expression 4. 

(3) Obtain a residual vector g'-Hg"0 given by a subtraction from the obtained rotated image information g'. 

(4) Obtain an estimated vector from the residual vector (g'-Hg"0) and HT 

(5) Find a new corrected vector by adding the estimated vector and the corrected vector 
40 (6) Repeat the above steps (2) through (6) using the newly estimated vector. 

[0076] In the above described method, called consecutive method, often, the calculation is ceased depending on 
the size of the residual vector found In the step (3). If the k-th calculation is represented by a suffix k, then the Jacobian 
method can be represented by the following Expression 10: 



Expression 10 g ^^^^ = Or + aH^(g'-Hgk) 



wherein a is called the damping coefficient, and detennines convergence and converging speed of the consecutive 
so method. The value of a is often determined by experience. The solution found by sufficient convergence by infinite 
number of repetitions in the Jacobian method satisfies the following Expression 11 : 



Expression 11 g^o = qL + aH^(g' - Hg^) 
[0077] Because the Expression 11 is satisfied, It is clear that the following Expression 12 is satisfied: 
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Expression 12 g' = Hg^ 

[0078] Now, having described technological philosophy about the digital wateimarlc infoimation detecting method 
5 according to the present invention, hereinafter, a digital watennark Information detecting method as an embodiment 
of the above will be described In detail with reference to a flowchart shown in Fig. 5. 

[0079] First, subject image infomfiatlon presumed to have been tampered and therefore is subjected to the digital 
watemnark information detecting method according to the present Invention is a still Image and this image infomiation 
preferably Is image Infomiation presumed to have been made by a rotating defonnlng operation. 
10 [0080] Further, preferably, source infomiation from another piece of Image infomiation, whteh is practically identical 
with or similar to the subject image infonmatlon, Is available. 

[0081] In other words, in the digital watemriark infomriation detecting method according to the present invention, in 
order to retrieve the digital watennark Infomiation embedded in the original image information, from image information 
which has Its digital watermark infomiation already deleted, altered or defomried by tampering, it is preferable that 
15 image information identical with or similar to the original image infomiation from a known source Is available. 

[0082] Further, in the digital watennark infomnatlon detecting method according to the present Invention, when the 
subject image intonnation is analyzed, IL lb piefiirauie that thera Is provided image information «naiy7ing means 1 
capable of identifying if the subject image Infomiation Is an enlarged/reduced Image infomriation or a rotated Image 
information. 

20 [0083] Specifically, as a means for identifying if a still Image obtained from the market is a rotated image or an 
enlarged/reduced image, there Is an "image edge detecting method" for example. By using this method described here 
below, it becomes possible to detenmine if the still image available from the market is a rotated or enlarged/reduced 

image. 

[0084] That is, in the Image edge detecting method, as shown in Fig. 7, the angle of rotation of the Image can be 
25 calculated by extracting an edge of the image by using a differential filter. 

[0085] On the other hand, when the edge cannot be detected by means of the image edge detecting method, the 
still image obtained from the market may be assumed to be an enlarged/reduced Image. 

[0086] Further, determination whether or not the still Image Is an enlarged/reduced image can be made by a method 
proposed in another patent application applied by the applicant of the present invention: In this method, the obtained 

30 image is Fourier transformed into a Fourier-space image, into which a predetennined reference Information signal is 
embedded at a predetennined location. Then the Fourier-space image Is reverse-Fourier-transfomried back into an 
Image-space image. Next, a plurality of enlarging/reducing rates are selected fomi a predetennined range of enlarging/ 
reducing rate, and the image-space image is enlarged/reduced by using each of the selected rates to obtain individual 
image of the selected sizes in the image-space. Then, each of the image-space images having different sizes of the 

35 image is Fourier transformed again into a Fourier-space image, and then in each of the Fourier-space image, the 
reference signal infomriation embedded into the image is detected to see if a frequency component of the reference 
signal information has been shifted or not. 

[0087] According to the present Invention, as has been described earlier, the subject image information obtained is 
an image which is already rotated and its digital watermark infomnatlon can no longer be readily legible: To this image 

40 information, first the estimated Image Infonnatlon vector g"0 (initial estimated image value) is set. Then, the estimated 
value is changed consecutively so as to minimize the square error value as given by the Expression 8, which is an 
expression of the en-or value between the subject image Infonnation vector g' and Hg"0 which represents the scale to 
measure the probability of the estimation given by the product of the estimated image value and the system matrix H. 
In such a way, by estimating the original image infomriation from the subject image infonnation, the digital watennark 

45 information is made legible. 

[0088] Preferably, the square error value determining means 6 used in the present invention is means which calcu- 
lates the least square enror value by using the affine inverse matrix or the numerical solution by means of the Jacoblan 
method. 

[0089] Referring now to the flowchart shown In Fig. 5, a basic steps of operation in the digital watennark Infonnation 
so detecting method according to the present Invention is as follows: 

[0090] Specifically, commencing the routine at the Start, Step {ST-1 ) Is performed to find the matrix H from the subject 
image infonnation obtained, and the transposed matrix HT is prepared. 

[0091] Here below, an example of how the vector H and its transposed matrix HT are prepared in the present case 
according to the present invention. 
55 [0092] Specifically, it is assumed that there Is a rotated still image as shown in Fig. 7. 

[0093] The size of the Image Is 4 x 4 pixels, and an angle of the rotation detected by the image edge detecting method 
is 15 degrees. 

[0094] From the conditions given above, the size of the vector H and its transposed matrix HT are 1 6 x 1 6 pixels. 
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[0095] Values of each element h(i, j) of the vector H given by the Expression 2 and 3 are listed in Table 1 to the 
second decimal place. 

[0096] The element h{i, j) represents connection between the pixel i (0 < i < 4 x 4) of the image after the rotation and 
the Image j (0 < j < 4 x 4) of the image before the rotation. 

[0097] As shown in Table 1 , if ail the elements h(i, j) take the value zero for the pixel i, this particular pixel does not 

have any relationship with any of the pixels of the image before the rotation. 

[0098] In other words, this particular pixel is connected with another pixel which is not in the.size of the image before 
the rotation. If the sum for the pixel i is not zero, then as shown in Expression 13: 



Expression 13 h ^-h +h 

= 1 

[0099] On the other hand, the transposed matrix HT of the vector H can be obtained by exchanging the row and the 
column of the vector H. 
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[0100] Next, in Step(ST-2) the original image information vector is estimated from the subject image infomnation 
vector g'. In this process, a first estimated image infonnatlon g"0 (initial estimated image value) is obtained, and then 
Step (ST-3) Is pert onned, In which the residual (g' - Hg"k) Is calculated as an error between the subject Image rnfonnation 
vector g' and the above Hg"0. 

[0101] It should be noted here that the estimated image infomnation g"0 (initial estimated image value) will be given 
a new value, as will be described later, every time the operation is repeated, and for this reason, generally, the estimated 
image infonnation vector value is expressed as g"k. 

[0102] Now, as a specific means for obtaining the initial estimated image value, It is possible to use a reverse- rotated 
Image obtained by the linear interpolation method of the affine transfonning method (or nearest-neighborhood method) 
as the initial estimated image value. The reverse-rotated image can be obtained for example, by changing the sign of 
the angle of rotation calculated in the above described method and putting this value in the Expression 2. 
[01 03] Thereafter, Step (ST-4) is executed, In which the square en^or I = (g' - Hg"k)T(g' - Hg"k) of the residual obtained 
in Step (ST-3) is calculated. 

[0104] Thereafter, Step (ST-5) is executed to detennine whether or not the square error I represents the minimum 
value. If Step (ST-5) detemilnes that the square error I Is the minimum value, then Step (ST-6) is executed to estimate 
the original Image infomnation vector g. From a resulted estimation, the digital watemnark Infonnation is retrieved by 
using an appropriate digital watemiark information means 10, and Is displayed In an appropriate displaying means 11 , 
which brings the routine to an END. 

[0105] On the other hand, if the Step (ST-5) detemnines that the square error I does not represent the minimum 
value, then Step (ST-7) is executed to calculate the correcting vector a HT(g' - Hg"k), and then Step (ST-8) is executed 
to obtain the reverse-rotated image vector expressed by g"k+1 = g^k-Kx HT(g'-Hg"k). 

[0106] Thereafter, the process goes back to Step (ST-3), in which the reverse-rotated Image vector g"k+1 is added 
to the residual value (g* - Hg"k), and then each of the steps following the Step (ST-3) is repeated. 

[01 07] As is clear from the description of the above example, as a more specific embodiment of the digital watennari< 
information detecting method according to the present invention, there is provided a digital watennark infonnation 
detecting method comprising: a first step of obtaining Image information; a second step of determining whether or not 
the obtained image Information obtained has been rotated or not; a third step of calculating an estimated initial image 
vector from the image infonnation which is found to have been rotated; a fourth step of calculating an estimated Initial 
image vector from the image infonnation; a fifth step of calculating a residual vector between the image vector in the 
image information and a value obtained by multiplying the estimated initial Image vector with the matrix; a sixth step 



15 



EP1 156 660 A2 



of calculating a square error of the residual vector; a seventh step of determining whether or not the residual vector 
square error value is a minimum value; an eighth step of calculating a reverse-rotated image when the residual vector 
square error is detemiined not to have the minimum value; a ninth step of adding the calculated reverse-rotated image 
to the value given by the estimated initial Image vector calculating means; a tenth step of repeating the step of calculating 

5 the residual vector square error value if need be; an eleventh step of estimating the original image infonnatlon from 
the residual vector square en'or value if the square error value is determined to be the minimum value; and a twelfth 
step of retrieving the digital watemnark information from the estimated original image Infonnation. 
[0108] Preferably, in the digital watermark information detecting method according to the present invention, the 
square error value determining means 6 is a means which calculates the least square error value by using the affine 

10 inverse matrix or the numerical solution by means of the Jacoblan method. 

[0109] The more specific embodiment of the digital watennark information detecting method according to the present 
invention will now be described in more detail, with reference to a flowchart shown in Fig. 6 
[0110] Specifically, in the digital watennark information detecting method according to the present Invention, when 
the routine is started, Step (S-1 ) Is executed to perform the first step of obtaining a subject Image Infonnation, and then 

15 Step (S-2) to detemnine whether or not the image infonnation obtained has been rotated. 

[0111] If the Step (S-2) determines that the obtained subject Image infonmation has not been rotated or has been 
defonned only by an enlarging/reducing defonning operation, then this routine will come to an end. However, If It is 
deienninea tnat the subject image infonnation has been subjected to a rotating defonning operation, then the routine 
goes to Step (S-3) to execute the third step of calculating the matrix H and the transposed matrix HT about the system 

20 matrix from the image information which is found to have been rotated, and then in Step (S^), the fourth step is 
executed to calculate the estimated initial Image vector g'O from the subject image information g'. 
[0112] Thereafter, in Step (S-5), determination is made If there is substitution data based on the reverse- rotated 
image infonnation vector g"k+1 to be described later. If the answer Is no, then Step (S-6) is executed to carry out the 
fifth step of calculating the residual vector (g' - Hg"k) between the image vector g' in the image infonnation and a value 

25 Hg"0 obtained by multiplying the estimated initial image vector g'O with the matrix H. 

[01 1 3] If the answer given by the Step (S-5) Is YES, then Step (S-7) is executed to carry out a 5*-th step of calculating 
a residual vector (g' -Hg" k+1 ) by using the substituted image information vector g"k+1 . 

[0114] Next, the routine goes to Step (S-8), where the sixth step is executed to calculate the square error I = (g' - 
Hg"k)T(g' - Hg-k) of the residual vector (g* -Hg"k), and then in Step (S-9), the seventh step is carried out to detennine 

30 whether or not the residual vector square error value I is the minimum value. 

[0115] If the Step (S-9) determines that the residual vector square error value I is not the minimum value, then routine 
goes to Step (S-1 0) to execute the eighth step of calculating the correcting vector represented by a HT(g' - Hg"k), and 
then in Step (S-11), the ninth step is executed to calculate the reverse-rotated image vector represented by g"k+1 = 
g"k + a HT(g' - Hg"k). Then, the routine goes back to Step (S-5), the tenth step is executed, in which the previous 

35 estimated image infonnation vector value g"0 or g"k is substituted by the newly calculated reverse-rotated Image in- 
formation vector g"k+1. 

[0116] Thereafter, the eleventh step is executed, in which steps from (S-4) through (S-11) are repeated depending 

on the need until the Step (S-7) detennines the cun-ent square error value I is the minimum, at which point the routine 
goes to Step (S-1 2), where the twelfth step is executed to estimate the original Image information g from the minimum 
40 square error value I. Then, in Step (S-1 3), the thirteenth step is executed to retrieve the digital watermark information 
from the estimated original image infonnation g, and finally, in Step (S-1 4), the step of displaying the digital watermaric 
information on specified displaying means is executed, bringing the routine to an end. 

[0117] Preferably, the repeat of the steps (S-5) through (S-11) is finished within five times for example, although 
there is no specific limitation to the number of repeats. 
45 [0118] Now, in order to describe the digital watennark infonmation detecting method in more specific details, simple 
model of image infonmation will be used and steps for carrying out the digital watermark infonnation detecting method 
will be detailed hereinafter. 

[0119] Specifically, an original still image as shown In Fig. 8, which has not been rotated, is used as the model. 
Assumptions are further made that the Image is rotated by an angle of 15 degrees as shown in Fig. 9, with a resulting 
50 size of the rotated image being 10x10 pixels. 

[0120] Grayed areas of the images shown In Fig. 8 and Fig. 9, are areas in which image values exist corresponding 
to non-zero values. 

[0121] For the above model of the still image, estimation was repeated, with an weight coefficient a in the Expression 
1 0 set to 1 .0. Results of the zero-th estimation (the initial value), 25-th, 50-th and 1 00-th estimations are shown re- 
55 spectively in Figs. 10 through 13. 

[01 22] Specifically, the square error 12 between the image g" and the original image g, and the square error 11 between 
the originally rotated image g' and the secondly-rotated image g' " = Hg" resulted from the re-rotation at the same angle 
In each of the sessions shown In Fig. 10 through Fig. 13 are respectively given by Expression 14 and Expression 15. 
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Expression 14 

i,=fe'-Hrr=is'-gt=-4l:fe-fn- 



10 

Expression 15 o i 

IS 

wherein M represents the size of the Image. Results obtained by the Expression 14 and the Expression 15 are shown 
in Fig. 14, Fig. 15 and Table 3. 

[0123] From the resultsshown in Fig. 14, Fig. 15andTable3, it is learned that the square error between the estimated 
20 image value and the original Image value becomes almost zero after about 25 times of estimation. 

[0124] Another aspect of the present Invention provides a recording medium storing a program for executing a digital 
watemriark infomnation detecting method, the method comprising: a first step of obtaining image infomnation; a second 
step of detemriining whether or not the obtained image information obtained has been rotated or not; a third step of 
calculating a matrix and a transposed matrix about system matrix; a fourth step of calculating an estimated initial image 
25 vector from the Image infomnation; a fifth step of calculating a residual vector between the image vector in the image 
information and a value obtained by multiplying the estimated initial image vector with the matrix; a sixth step of cal- 
culating a square error of the residual vector; a seventh step of determining whether or not the residual vector square 
error value is a minimum value; an eighth step of calculating a reverse- rotated image If the residual vector square error 
is detennined not to have the minimum value; a ninth step of calculating a reverse-rotated image from the con-ecting 
30 vector; a tenth step of substituting the estimated image vector value with the calculated reverse-rotated image infor- 
mation; an eleventh step of repeating the step of calculating the residual vector square error value as necessary; a 
twelfth step of estimating the original image Information from the minimum square en-or value if the residual square 
error value is detennined to be the minimum value; and a step of retrieving the digital watermark information from the 
estimated original image Information. 

35 

Table 3 



Number of estimations 


ll 


l2 


0 


0.030703 


0.019519 


25 


0.000013 


0.000214 


50 


0.000001 


0.000021 


100 


0.000000 


0.000001 



[0125] Next, an embodiment of the above aspect of the present invention will be described in detail. 

[01 26] Specifically, description will cover, with reference to the accompanying drawings: an original image deforming- 

method detecting device 300 which carries out the original Image defomning-method detecting method according to 

the present invention: an original-image-size detecting device 100 which canries out the method for detecting the size 
of the original image; and a digital-watermark detecting device 200 which carries out the digital-watemriark detecting 
method. 

[0127] First, Fig. 16 is a block diagram showing a constitution of the original image defomiing-method detecting 
device 300 which canies out the original image def omning-method detecting method according to the present invention. 
The figure shows the original Image defonning-method detecting device 300 comprising: first Fourier transfonning 
means 3 which Fourier transforms an obtained Image 22 into a first Fourier-space Image 20; reference infomnation 
signal embedding means 24 which embeds a predetermined reference infomnation signal S at a predetennined location 
P of the Fourier-space image 20 which Is prepared by Fourier transforming the obtained image; reverse Fourier trans- 
fonning means 25 which reverse-Fourier-transfonms the Fourier-space image 20 provided with the reference infomria- 
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tion signal S Into an image-space Image 30; enlarging/reducing rate storing means 26 which stores a plurality of en- 
larging/reducing rates D (D1 , D2, D3,...Dn) for changing the size of the Image-space Image 30; Image* size changing 
means 27 which sequentially selects desired enlarging/reducing rates D (D1 , D2, D3,...Dn) from the enlarging/reduci 
ng rate storing means 26, and sequentially and individually changes the image size of the image-space image 30 by 

5 individually using each of the selected enlarging/reducing rates D1 , D2, D3,...Dn; second Fourier transforming means 
28 which Fourier transforms each of the differently-sized Image-space images 30 into second Fourier-space Images 
40; frequency component value detecting means 29 which detects a frequency component of the reference Information 
signal S from each of the second Fourier-space images 40; frequency component value arraying means 20 which 
arrays a plurality of frequency components obtained from each of the second Fourier-space images 40 into a prede- 

10 tennined sequence according to the enlarging/reducing rates D (Dl, D2, D3,...Dn); outstanding frequency component 
value detennlning means 31 which detemnlnes whether or not there Is a frequency component value outstanding from 
surroundingfrequency component values; and original-image enlarging/reduclng-operatlon Identifying means 41 which 
determines, on the basis of an output value outputted from the outstanding frequency component detenmining means 
31 , that the subject image is a result of an enlarging/reducing defomning operation perfomried to the original image. 

15 [01 28] Therefore, needless to say, according to the present embodiment, there is a case in wh ich the obtained subject 
still image 22 obtained is detemriined not to have been rotated nor enlarged/reduced. 

[0129] Preferably, the original image defonnlng-method detectinq device 300 according to the present invention is 
provided with noise removing means 33 which removes a predetermined noise, i.e. a background value, from the 
output data from the frequency component value arraying means 20 when the output from the frequency component 
20 value arraying means 20 is searched by the outstanding frequency component value determining means 31 for an 

outstanding frequency component value. 

[0130] Further, preferably, the original Image defomning-method detecting device 300, which has to start operation 
without any information about the kind of deforming operation performed to the obtained still image 21 , is provided for 
example, with simple detecting means such as rotating defomning operation detecting means 1 7 for detennining wheth- 

25 er or not the obtained still image 22 has been rotated. 

[01 31 ] There are already several publicly known specific methods for determining whether or not the subject image 
obtained has been rotated or otherwise defomned. As an easier method of these, the "image edge detecting method" 
which will be described here below can be used conveniently as a method of detecting if a still Image obtained from 
the market is a rotated image of not. 

30 [0132] Specifically, according to the image edge detecting method, as shown in Fig. 22, an angle of rotation of the 
image can be calculated, by using a differential filter to find out an edge of the image. 

[0133] On the other hand, if the edge cannot be found by the above image edge detecting method, the still image 
obtained from the mari<et can be presumed to have been subjected to the enlarging/reducing deforming operation or 

another defomning operation. 

35 [0134] Therefore, according to the present embodiment, the still Image 22 obtained Is first checked by the rotating 
deforming operation detecting means 1 7. If the Image Is found to be a rotated image, then the result of the check can 
be displayed in a displaying means 23, and the rest of the process can be cancelled. If the rotating deforming operation 
detecting means 1 7 determines that the obtained still Image 22 has not been rotated but has been otherwise deformed 
by methods including the enlarging/reducing defomning operation, then the series of means starting from the first Fourier 

40 transforming means 3 will be carried out In order to find if the obtained still image 22 has been enlarged/reduced. 
[0135] On the other hand, according to the present Invention, the first and the second Fourier transforming means 
23, 28 may be provided individually or provided as a single means commonly used for both functions. 
[0136] Further, in the original image defomning-method detecting device 300 according to the present embodiment, 
It Is possible to display the detemnlnation made by the original-image enlarging/reducing deforming operation identifying 

45 means 41 in an appropriate displaying means 42. 

[01 37] It should be noted here that a component indicated by reference numeral 1 8 in Fig. 1 6 is a central processing 
unit (CPU) which integrally controls each of the means constituting the original image deformlng-method detecting 
device 300 according to the present invention as well as two other devices to be described hereafter, i.e. the original- 
image-size detecting device 100 and the digital -watermaric detecting device 200. 

50 [0138] Next, constitution of the original-image-size detecting device 1 00 and the digital-watermark detecting devtee 
200 according to the present invention will be described with reference to Fig. 16. 

[0139] Fig. 1 6 is a block diagram showing a constitution of an embodiment of the devices 1 00 and 200 for carrying 
out the original image size detecting method and the digltal-watemiark detecting method according to the present 
invention. The figure shows the device 100 which detects the size of the original image from the obtained Image 22. 
55 The device comprises: first Fourier transforming means 23 which Fourier transfomns an obtained Image 22 Into a first 
Fourier-space Image 20; reference infomnation signal embedding means 24 which embeds a predetennined reference 
Infomiation signal S at a predetemnined location P of the Fourier-space image 20 which is prepared by Fourier trans- 
forming the obtained image; reverse Fourier transfomning means 25 which reverse-Fourier-transfomns the Fourier- 
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space image 20 provided with the reference information signal S into an image-space image 30; enlarging/reducing 
rate storing means 26 which stores a plurality of enlarging/reducing rates D (D1 , D2, D3,...Dn) for changing the size 
of the Image-space image 30; image-size changing means 27 which sequentially selects desired enlarging/reducing 
rates D (D1 , D2, D3,...Dn) from the enlarging/reducing rate storing means 26, and sequentially and individually change 

5 the image size of the image-space image 30 by individually using each of the selected enlarging/reducing rates D1 , 
D2, D3,. ..Dn; second Fourier transforming means 28 which Fourier transforms each of the differently-sized image- 
space images 30 into second Fourier-space images 40; frequency component value detecting means 29 which detects 
a frequency component of the reference infonnation signal S from each of the second Fourier-space images 40; fre- 
quency component value arraying means 20 which arrays a plurality of frequency components obtained from each of 

10 the second Fourier-space images 40 into a predetennlned sequence according to the enlarging/reducing rates D (D1 , 
D2, D3,...Dn); outstanding frequency component value detennining means 31 which determines whether or not there 
is a frequency component value outstanding from surrounding frequency component values; and original-image size 
identifying means 32 which detemriines, if the outstanding frequency component is found, that the enlarging/reducing 
rate Dn which corresponds to the outstanding frequency component value is the enlarging/reducing rate D which was 

IS used for malcing the obtained image 22 by changing the size of the original image 2. 

[0140] Preferably, the original-image-size detecting device 100 according to the present invention is provided with 
noise removing means 33 which removes a predetemiined noise, i.e. a background value, from the output data from 
the neMueiiuy uomponeni vaiue arraying means 20 when the output from the frequency component value arraying 
means 20 is searched by the outstanding frequency component value detenmining means 31 for an outstanding fre- 

20 quency component value. 

[0141] On the other hand, the digital-watermark detecting device 200 according to the present invention comprises 
all of the constitution of the original-image-size detecting device 100 shown in Fig. 1 , and further comprises: original 
image reproducing means 14 which reproduces an estimated original image 21 ' by applying to the obtained image 22 
a reciprocal number 1/D of the enlarging/reducing rate Dn which is the rate selected by the original-image size iden- 

25 tifying means 32 as the rate believed to have been used in the enlarging/reducing defomiing operation perfonned to 
the original image 21 in order to make the input image 22; and digital watermark information detecting means 1 5 which 
detects presence or absence of the digital watenmark infomnation in the estimated original image 21 ' reproduced by 
the original image reproducing means 14. 

[0142] Further, preferably, the original-image-size detecting device 100 and the digital-watemnark detecting device 

30 200 according to the present invention are provided with subject-image defonning-operatlon detecting means 1 7 whtoh 
detemiines whether or not the obtained still image 22 is made by an enlarging operation or a reducing operation. 
[0143] On the other hand, according to the present invention, the first and the second Fourier transfomrilng means 
23, 28 may be provided individually or provided as a single means commonly used for both functions. 
[01 44] Further, preferably, in the original-image-size detecting device 1 00 and the digltal-watemiaric detecting device 

35 200 according to the present invention, there is provided a digital watennark infonnation coincident retrieving means 
16 which detemiines, when the digital watemnark infomnation detecting means 15 has retrieved a digital watennaric 
from the estimated original image 21', if the digital watemnark found in the reproduced original image 21' is identical 
with or similar to infomnation stored in original-image digital-watemriark storing means 20 which stores the digital wa- 
temnark information peculiar to the standard original image 21 . Further, preferably, there is provided displaying means 

40 24 which displays a result of the retrieve perfonned by the digital watennark information coincident retrieving means 16. 
[0145] Now, a basic operating principle of the original image size detecting method and the digital-watennark de- 
tecting method according to the present invention will be described in detail with reference to Fig. 17 through Fig. 24. 
[0146] First, conventional method of embedding a digital watennart< into a desired original image can be classified 
roughly into two kinds, one kind being embedment into the image space while the other kind being embedment Into 

45 the Fourier space. 

[0147] Specifically, according to the image space technique, the digital watennark infomnation is embedded directly 
into image values. On the other hand, according to the Fourier space technique, the image values are Fourier trans- 
formed, and the digital watennark infonnation is embedded Into the Fourier values of the Image. 
[0148] Each of the methods has its own advantages. 

50 [0149] For example, if digital watemnark information A having a predetemnined frequency value is embedded into 
the image-space image 22 or 30 as shown in Fig. 1 7(A), and if this image-space image if Fourier transformed into the 
Fourier-space image 20, the frequency of the embedded digital watemnark infomnation A is diffused generally unifomnly 
into the Fourier space as shown in Fig. 17(B), becoming unrecognizable as the digital watennark infomnation. 
[01 50] On the other hand, if the Fourier-space image 20 which has digital watermark infonnation B embedded within 

55 is reverse- Fourier-transformed into the image-space image 30 as shown in Fig. 1 8 (B), energy (frequency component 
values) of the digital watennark infonnation B is diffused into the entire image-space as shown in Fig. 18(A), with less 
deterioration in the quality of the original image. 

[0151] On the other hand, as has been described above, digital watermark infonnation embedded in the Fourier- 



19 



EP1 156 660 A2 



space image 20 has the following characteristics: 

[0152] First, consideration is made for one-dimensional signal, taking an example shown in Fig. 19(A), in which the 
one-dimensional signal is expressed as one-dimensional time series having a period of 8P (P: pixel) and a length of 
256P. If a 128-point sample having the enlarging/reducing rate of 1 .0 (i.e. a series which is not enlarged nor reduced) 

5 is Fourier transformed as shown in Fig. 1 9(B), a resulting frequency is given by 1 28P/8T = 1 6 as shown In Fig. 1 9(C). 
[0153] Specifically, Fig. 19(C) shows the state of the frequency component of the sample taken, without changing 
the size of image, from the one-dimensional time series having the period of BP (P: pixel) and the length of 256R 
[0154] Next, Fig. 20(A) through Fig. 20(C) show a result of 128-pixel sampling from a similar one-dimensional series 
after the series is reduced at a rate of 0.8. Fig. 20(B) shows the waveform of the sample. 

10 [0155] Further, If this series is Fourier transfomied, it is leamed that the frequency of the series becomes greater 
than 16 as shown in Fig. 20(C). 

[0156] Further, Fig. 20(A) shows a result of 128-pixel sample taken from the similar one-dimensional series after the 
series is enlarged at a rate of 1 .2. Fig. 20(B) shows the wavefonn of the sample. 

[0157] Further, if this series is Fourier transfomied, it is learned that the frequency of the series becomes smaller 
IS than 1 6 as shown in Fig. 20(C). 

[0158] As is clear from the above results, if a signal having a certain frequency Is enlarged/reduced at a predetermined 
enlarging/reducing rate and then re-sampled, then, as shown in Fig. 22, the frequency of the signal becomes greater 
or smaller than the original frequency. 

[01 59] It should be appreciated to note here that the unit of measurement of the vertical axis in Fig. 22 is the intensity 
20 of the frequency component. 

[0160] Specifically, it is learned that the location of the watermark signal embedded In the Fourier space will shift 
after the enlarging/reducing defonning operation. 

[0161] Next, consideration is made to a case of two-dimensional image, in which the digital watennark infomnation 
is embedded into the Fourier-space image prepared by Fourier transfonning the image. A two-dimensional images as 

25 shown in Fig. 23(A) and Fig. 23(B) will be used as a model. 

[0162] The images shown in Fig. 23(A) and Fig. 23(B) respectively had a size of 256 x 256p, being a gray-<;olored 
image of the Image value of 128. This image was divided into four pieces of block Image each having the size of 1 28p 
X 128p. Each of the block Images were Fourier transformed, and then a total of eight predetemnined reference infor- 
mation signals S each having, for example, a height of 20 (8-blt image) and a thickness of 1 pixel, were disposed at 

30 each of the predetemnined locations P in each of the blocks in the Fourier transfonned image. 

[0163] It should be noted here that a gray symbol placed in an upper left corner portion of the Images represents a 
zero-frequency component of the original image in Fig. 23. 

[0164] Thereafter, each of the four block-images carrying the reference infonnation signals S embedded within was 
reverse-Fourier-transfonned back into an image-space image. Then, from this image-space image, each of the 128 x 
35 I28p blocks containing the reference infonnation signals S is cut out, and Fourier transfonned again. After this oper- 
ation, each of the locations of the reference information signals S indicates the original location P represented by gray 
markings in Fig. 23(A) and Fig. 23(B). 

[0165] However, it was found that if the image-space image having the reference information signals S embedded 
within Is first reduced, for example by the enlarging/reducing rate of 0.8 before the second Fourier transfomnation, then 

40 the frequency component values of the reference information signals S will shift inwardly of the original location P 
where the reference information signals S are first embedded, as indicated by black maricings P' in Fig. 23(A). 
[0166] Likewise, it was found that if the image-space image having the reference infonnation signals S embedded 
within is first enlarged, for example by the enlarging/reducing rate D of 1 .2 before the second Fourier transformation, 
then the frequency component values of the reference information signals S will shift outwardly of the original location 

45 p where the reference infonnation signals S are first embedded, as indicated by the black markings P' in Fig. 23(B). 
[01 67] From the above, it is clear that the location of the watennark signal embedded into a Fourier-space image of 
a predetermined image will shift laterally or outwardly, or toward the center when the image is enlarged or reduced. 
[01 68] The original image size detecting method and the digltal-watennark detecting method according to the present 
invention use this behavioral characteristics of predetemnined infomiation embedded into the Fourier-space image, 

50 thereby making possible to detect the sizes of the original image and digital watermark infonnation embedded into the 
original image. 

[01 69] A bask: technical process in the original Image size detecting method and/or the digital-watemrtartc detecting 
method is an encoder/decoder process to detect the size of the original image by using the Fourier-space, as an 
application of the above described principle. 
55 [0170] Specifically, in the encoding operation, for example: 

1 ) The obtained subject image is divided into blocks each having a size of 128 x 128, and if there is any remaining, 
then zero data is added to make up the block size. 
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2) Each of the block images is Fouriertransformed and predetermined reference information signal S is embedded 
into a predetenmined location P. The intensity of the reference infonnation signal to be embedded is adjusted In 
accordance with the height of the image frequency component at the location P. 

3) Each of the block images carrying the reference Infonnation signal S is reverse-Fourier-transformed back Into 
5 the original block position of the obtained subject Image. 

[0171] The decoding operation thereafter is: 

1 ) The enlarging/reducing defomnlng operation is performed to the subject Image after the subject image is trans- 
10 formed back the Image-space Image. When the enlarging/reducing rate is between 0.5 and 1 .0 for example, the 

operation is the reducing operation, whereas when the enlarging/reducing rate is between 1 .0 and 2.0, the operation 
is the enlarging operation. 

It should be noted that the size of each image to be enlarged/reduced Is 128 x 128. 

2) A plurality of enlarging/reducing rates are used to prepare the plural sets of enlarged or reduced block Images. 
15 These block Images are Fourier transformed again Into the Fourier-space images, and from each of the Fourier- 
space images, frequency component is picked up individually at the embedding location P of the reference Infor- 
mation signal S in the image. 

In the above operation, preferably the frequency component value is an average value. 
By Integrating or arraying the obtained frequency component values In an ascending or descending order in 
20 accordance with the size of enlarging/reducing rate, a graph as shown in Fig. 24 Is obtained. 

3) From the graph shown in Fig. 24, as necessary, noise component such as influence from sunroundings (influ- 
ences by integration of the original image frequency component at each location of embedment) is removed. There- 
after, a series of graphs of the frequency component values are checked if there Is a specific frequency component 
value outstanding from sun^ounding values. 

25 A reason why the above operation according to the present invention creates the frequency component value 

outstanding from surrounding frequency component values In the series of graphs of the frequency components 
values may be explained as follows: 

Specifically, if an image carrying an embedded digital watermark information is enlarged/reduced, the em- 
bedded frequency component (the reference infonnation signal) becomes out of the original position. Thus, when 

30 the size of the image is varied by using a plurality of enlarging/reducing rates, and the frequency component value 
is detected at the location where the reference information signal was embedded, then the frequency component 
value is found to be the maximum value If the image size is equal to the original Image size. 

4) If the outstanding frequency component value is found, detemninatton is made that the value of the enlarging/ 
reducing rate D corresponding to the outstanding frequency component value is the enlarging/reducing rate D 

35 used when the obtained subject image was made from the original image by changing the size of the Image. 

[0172] In other words It becomes possible by the method described above, to detennine the unknown size of the 
original image from which the obtained subject image was made. 

[0173] In other words, from the graphs in Fig. 23, it is detennined that the obtained subject image is an image made 
40 by enlarging the original image by the enlarging/reducing rate D of 1 .2. 

[0174] Therefore, by perfonning a reverse operation to the obtained subject image, or according to the above ex- 
ample, by reducing the obtained subject image at the enlarging/reducing rate D of 1/1 .2, the estimated original image 
1' can be reproduced. 

[0175] Thereafter, the digital watennark infomriation is retrieved from the reproduced original Image 21' by using a 
45 publicly known method, and the retrieved digital watermaric infonnation is compared with the digital watennari< infor- 
mation carried by the original Image 21 whose source is already known. By these operations, it becomes possible to 
Identify if the obtained subject image comes from the same source or not. 

[0176] Therefore, by using the original image size detecting method and the digital-watermari< detecting method 
according to the present invention, it becomes possible to confinn the source of an image if the image is made by 
so tampering a known original image, i.e. by performing the enlarging/reducing defonning operation, thereby deleting or 
making Illegible the digital watermaric information initially embedded In the original image. Thus, It becomes possible 
to detennine if the obtained subject image is made by tampering the original Image from a known source or Is an 
imitation thereof created separately. In addition, It also becomes possible to detennine at which level of the market the 
tampering was made. 

55 [0177] Hereinafter, description will be made in further detail for the embodiment of the original image defonning- 
method detecting method, the original-image-size method and the digital-watermark detecting method that use the 
original image deforming-method detecting device 300, the original-lmage-size detecting device 100, and the digital- 
watemnaric detecting devtoe 200 according to the present invention. 
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[0178] Specifically, first, a basic constitution of the original image defomning-metiiod detecting nnethod according to 
the present invention comprises the steps of: embedding a predetemnined digital watermark information S at a prede- 
termined location of a Fourier-space image 20 prepared by Fourier transformation of an obtained subject Image 22; 
changing the Fourier-space Image 20 baci< Into an Image-space image by reverse-Fourler-transfomnation; Individually 

5 changing the image size of the image-space image 30 by using a plurality of enlarging/reducing rates (D1 , D2, D3,... 
Dn) selected from a predetermined range of the enlarging/reducing rates D; preparing Fourier-space images 40 by 
Fourier transfomiing each of the differently-sized image-space images 30; detecting a frequency component of the 
reference Infomnation signal S embedded In advance, from each of the Fourier-space images 40; arraying a plurality 
of frequency components of the reference information signal S obtained from each of the Fourier-space Images 40 

10 into a predetemnined sequence according to the size of the enlarging/reducing rates D1 , D2, D3,...Dn; and checking 
to see if there is a frequency component value outstanding from surrounding frequency component values: and if there 
is a certain frequency component value outstanding from the surrounding frequency component values, then the subject 
image 22 is detemnined to have been made by enlarging/reducing an original image. A basic constitution of the original 
Image size detecting method according to the present invention comprises the steps of: embedding a predetemnined 

15 digital watermark infomnation Sat a predetennined location of a Fourier-space image 20 prepared by Fourier transfor- 
mation of an obtained subject image 22; changing the Fourier-space image 20 back Into an image-space image by 
reverse-Fourier transformation; Individually changing the image size of the image-space image 30 by using a plurality 
of enlarging/reducing rates (D1 , D2, D3,...Dn) selected from a predetermined range of the enlarging/reducing rates D; 
preparing Fourier-space images 40 by Fourier transforming each of the differently-sized image-space images 30; de- 

20 tecting a frequency component of the reference Infomnation signal S embedded In advance, from each of the Fourier- 
space Images 40; arraying a plurality of frequency components of the reference Infomnation signal S obtained from 
each of the Fourier-space images 40 into a predetermined sequence according to the size of the enlarging/reducing 
rates D1 , D2, D3,...Dn; and checking to see if there is a frequency component value outstanding from surrounding 
frequency component values: and If there is a certain frequency component value outstanding from the surrounding 

25 frequency component values, then the subject Image 22 is detemnlned to have been made from the estimated original 
Image 21'by changing the size of the image by the enlarging/reducing rate Dcon'esponding to the frequency component 
value outstanding from the surrounding frequency component values. 

[01 79] Specifically, the above two aspects of the present invention makes possible the following options for example, 
for a person who is an owner of copyright to a certain Image carrying a predetermined embedded digital watermari< 

30 information and being sold in the market. (In this case, the image whk;h is being sold with the predetemnined digital 
watemnari< infomnation embedded, is the original image 21 . The original image 21 is an Image whose source Is already 
known .) If there is marketed another image 22 which Is close to, similar to or looks Identical with the original image 21 , 
yet the source of the image 22 is unknown, and further, the digital watermark information is deleted or made illegible, 
and therefore a presumed original image 21' (hereinafter called presumed original image 21') cannot be confimned as 

35 the true source, then the image 22 may be brought in as the subject image 22, and by testing the subject image 22 
using the original image size detecting method and the digital-watennark detecting method according to the present 
Invention, It becomes possible to confirm if the presumed original image 21 is the true original image or not. 
[01 80] Therefore, In the present invention, first, it is necessary that the original image 21 is an Image whose source 
is already known. 

40 [0181] Further, in the present invention, the obtained image 22 is an image which Is generally similar to the original 
image 21 , and whose digital watemnark Infomnation Is deleted or illegible. 

[0182] Still further, according to the present Invention, the original image 21 and the obtained image 22 are still 
images. 

[0183] On the other hand, according to the original-image size-detemnining method and the digital-watennark de- 
45 tecting method according to the present invention, the subject image infomnation 22 presumed to have been tampered 
is a still image, and further, this image information is presumed to be made by enlarging/reducing the estimated original 
image infomnation 21'. 

[0184] Therefore, according to the present invention, when the subject Image information 22 is analyzed as shown 
in Fig. 16, operation detecting means 17 which determines whether the subject image 22 was enlarged/reduced or 
50 rotated. 

[0185] Specifically, It is preferable to utilize the above described an-angement. 

[0186] According to the present Invention, as has been described above, determination is first made as whether or 
not the obtained subject image 22 is a result of rotating deforming operation. If the obtained subject image 22 is de- 
temnlned to be the result of rotating defomning operation, the rest of the process is cancelled. If the subject image 22 
55 Is detemiined not to be the result of rotating deforming operation, then the rest of the process Is executed to determine 
if the subject Image 22 Is a result of the enlarging/reducing defomilng operation. 

[0187] In other words, according to the present invention, the subject image 22 presumed to have been tampered 
thereby had its digital watemnark information made illegible is analyzed to detemnine if the subject image 22 is enlarged/ 
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reduced, rotated or otherwise deformed. 

[0188] The range of enlarging/reducing rate D used in the present invention is not particularly specified. Preferably 
however, a range is from 0. 5 through 2.0 as a practical range. 

[0189] Preferably, such infomnation about the enlarging/reducing rate as mentioned above Is stored in advance, in 
5 enlarging/reducing rate storing means 26. 

[0190] Further, preferably, according to the present invention, the detected frequency component value is an average 
value. Still further, when finding a peak value of the frequency component, it is preferable that background value is 
removed from all of the detected frequency components, and for this particular purpose, it is also preferable to provide 
noise removing means 33. 

10 [0191] The digital-watennark detecting method according to the present invention uses the finding made by the 
above described original image size detecting method as to what value was used as the enlarging/reducing rate D 
when the subject image 22 was made by enlarging/reducing the original image 21' whose source is unknown. Specif- 
ically, the estimated original image 21' is reproduced by applying the reciprocal number 1/D of the obtained enlarging/ 
reducing rate D to the obtained image 22, and the estimated original image 21' is searched for the watemiark infor- 
ms matlon. 

[01 92] As a method for putting back the obtained subject image 22 Into the estimated original image 21 ', for example, 

u publiCi'y Icnovvn method o-ch 3s t.h2 Hffirs trsnsforming method m^y u'?'^'^ 

[0193] Now, a more detailed embodiment of the above described defonnati on -method detecting method, original 
image size detecting method and digital-watermark detecting method according to the present invention will be de- 

20 scribed hereinafter with reference to a flowchart In Fig. 25. 

[01 94] Specifically, when the original image size detecting method according to the present invention is started, Step 
ST-1 is executed to perform a first step of obtaining the subject image 22 which looks similar to the original image 21 
from a known source but whose digital watermark information is not legible. At the same time, preferably Step ST-1 01 
is also executed to detemnine if the subject image 22 is a rotated image or not: if the answer is YES, then the process 

25 ends but if the answer is NO, then the next step is perfomied. 

[0195] Next, in Step ST-2, a second step is executed to Fourier transfonn the obtained subject image 22 into a first 
Fourier-space image 20. Then, Step ST-3 is executed to perfomri a third step of embedding predetemnined reference 
infomiation signal S at a predetermined location P of the first Fourier-space image 20. 

[0196] Thereafter, the process goes to Step ST-4, where a fourth step of reverse- Fourier-transfomning the Fourier- 
30 space image 20 provided with the reference infonnation signal S into an image-space image 30. Then, in Step ST-5, 
a fifth step Is executed to individually change the size of the image-space image 30 by using a plurality of enlarging/ 
reducing rates D1 , D2, D3,...Dn selected from the enlarging/reducing rates D (D1 , D2, D3,...Dn) within a predetemnined 
range stored in the rate storing means 26. 

[0197] Next, the process moves on to Step ST-6, where a six step of Fourier transfomriing each of the differently- 
35 sized image-space images 30 into a plurality of second Fourier-space images 40. Thereafter, in Step ST-7, a seventh 
step of detecting a frequency component of the reference infonnation signal S embedded in the image 20 from each 
of the Fourier-space images 40. 

[0198] Thereafter, in Step ST-8, an eighth step is performed to array the detected frequency components of the 
reference infonnation signal S in a sequence by integration, or according to the size of the enlarging/reducing rates 

40 D, to obtain a wavefomn as shown in Fig. 25. 

[01 99] Next, the process moves on to Step ST-9, which detennines whether or not there Is a frequency component 
value corresponding to any of the enlarging/reducing rates Dn outstanding from surrounding frequency component 
values. If the answer is NO, then the process is brought to an END, with a detemnination that the subject image 22 is 
presumed not to have been rotated nor enlarged/reduced, but have been otherwise deformed. 

45 [0200] On the other hand, if the answer In Step ST-9 is YES, i.e. if there is a frequency component value outstanding 
from the surrounding frequency component values, then Step ST-90 detennines that the subject image 22 has been 
enlarged/reduced, and if needed, the determination is displayed in an appropriate displaying means 54, completing a 
ninth step of the original image defonning-method detecting method according to the present invention. 
[0201 ] On the other hand, in an embodiment of the original-image size-detennining method according to the present 

50 invention, as shown in a flowchart in Fig. 25, the same steps as in the above described original image defomning- 
method detecting method are executed from ST-1 through ST-9. From ST-9, the process moves, via ST-90, or directly 
to ST-91 to execute a 9a-th step, where it is detemnined that the enlarging/reducing rate Dn corresponding to the 
outstanding frequency component value is the enlarging/reducing rate Dn by which the size of the estimated original 
image 21' whose source is unknown was changed to make the obtained subject image 22. 

55 [0202] It should be noted here that in the Step ST-9, when finding a peak value of the frequency component. It Is 
preferable that background value is removed from all of the detected frequency components. 
[0203] The steps of operation described above are the basic constitution of the original-image size-detennining meth- 
od according to the present Invention. 
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[0204] In the digital-watermark detecting nnethod according to the present invention, the same steps as in the above 
described original image size detecting method are executed from the first step through the 9a-th step. Thereafter, the 
finding made by the Step ST-9 of the original Image size detecting method as to what value was used as the enlarging/ 
reducing rate D when the subject Image 22 was made from the estimated original Image 21 ' Is utilized. Specifically, a 

5 tenth step Is executed In Step ST-1 0, where the estimated original image 21' Is reproduced by applying the reciprocal 
number 1/D of the obtained enlarging/reducing rate D to the obtained image 22, and in Step ST-11 , an eleventh step 
is executed, where the estimated original image 21' whose source in unknown is searched for watemnaric infomriation. 
[0205] Thereafter, the step moves onto Step ST-1 2, where the digital watermark information obtained from the es- 
timated original image 21' whose source is unknown is compared to the digital watennark information of the original 

10 image 21 whose source Is already known and stored in digital watermari< infomriation storing means 56 shown In Fig. 
16, to detemiine if the two watermarks are identical with each other 

[0206] If the answer given in the Step ST-1 2 is NO, then the estimated source image 21' of the obtained subject 
image 22 can be presumed to be an imitated image which is not directly related to the original image 21 whose source 
Is already known. If the answer given in the Step ST-1 2 Is YES, then It can be presumed that the estimated source 
15 image 21' was made by tampering, i.e. by enlarging/reducing the original image 21 from the known source, and by 
deleting the digital watemnari^ Infomriation. 

[0207] Next, a more detaiiea example of ihe above uesoriijec! origins! Irr^agc dcfcrrrjirig-method detecting mfithod 
according to the present invention will be detailed with reference to Figs. 26 through 31 . 

[0208] Now, an assumption Is made that a gray image, as shown In Fig. 26(A), having a pixel value of 128 and a 

20 size of 256 x 256 pixels is obtained as the subject image 22. 

[0209] First, as shown In Fig. 26(A), the gray subject image 22 Is divided Into blocks each having a size of 1 28 x 1 28. 
[0210] Next, as shown In Fig. 26(B), each block image is Fourier transfomied, and a frequency component having 
an intensity of about five units is embedded as reference Infonnation signal, into predetemiined locations (eight loca- 
tions according to the present example). 

25 [021 1] More specif bally, the frequency component having the predetemiined Intensity Is embedded as the reference 
Infomriation signal at the following eight address locations of (8, 16), (16, 8), (112, 16), (120, 8), (8, 112), (16. 120). 
(112, 120), and (120, 112). 

[0212] Further, by reverse-Fourier-transfomnlng each of the block images, the reference infomriation signal Is em- 
bedded into the original image. 

30 [0213] As shown, in the above, the reference Information signal is embedded in the Fourier space. When the image 
is reverse-Fourier-transfomned Into the image space, energy of the embedded digital watermark Infomriation Is diffused 
Into the Image space, making the stains added to the original image less visible. 

[0214] Thereafter, as shown in Fig. 26(C), the image (a) in which the reference Information signal Is embedded is 
reduced by the rate t\ =1 .2 (y\ : Size of original image/Size of reduced image) to obtain an Image (b). 
35 [0215] Next, as shown in Fig. 26(D), as has been described eariier, a block of partial image (c) having a size of 128 
X 128 is cut out from the reduced image (b) and Fourier transfonned. It was learned that as shown In Fig. 27, the 
locations each represented by • (black circle symbol) at which the reference Infomriation signal was first embedded 
moves inward to places represented by ▲ (black triangle symbol). 

[0216] It should be noted here that# (black circle symbol) shown at an upper left corner in Fig. 27 represents zero- 
40 frequency component of the original image. 

[0217] Next, as shown In Fig. 26(D), the image In which the reference Infomriation signal is embedded Is enlarged 
by the rate t) =0.8 (i^ : Size of original Image/Size of reduced Image) to obtain an image (d). 

[0218] Thereafter, a block of partial image (e) having a size of 128 x 128 is cut out and Fourier transformed. It was 
learned that as shown in Fig. 27, the locations each represented by • (black circle symbol) at which the reference 
45 infomriation signal was first embedded moves outward to places represented by# (black diamond symbol). 

[0219] It should be noted here that# (black circle symbol) shown at an upper left corner In Fig. 27 represents zero- 
frequency component of the original image. 

[0220] Next, the size of the Image obtained from the market is changed by rates selected from the range of 0.5 
through 1 .5 as shown In Fig. 28. (According to the present example, the same rate Is applied in both directions along 

50 the x axis and the y axis.) 

[0221] Then, as shown In Fig. 26(C) and Fig. 26(D), an upper- most left block (e) of Image portions having the size 
128 X 128 was cut out from each of the enlarged and reduced images, and then Fourier transfonned. 
[0222] Next, for the Fourier-transformed block images, an average value of the eight frequency component values 
shown In Fig. 26(B) are calculated, to obtain the average frequency component value of the eight points for each of 

55 the images prepared by the different enlarging/reducing rates selected from the range from 0.5 through 1 .5 as shown 
In Figs. 29, 30 and 31. 

[0223] Thereafter, from graphs of the frequency component average values obtained from the above operation, a 
point outstanding from surrounding frequency component values Is detected, through which detemnlnation can be 



24 



EP1 156 660 A2 



made whether or not the obtained image is enlarged/reduced, or it becomes possible to calculate the rate used for the 
enlarging/reducing defomning operation. 

[0224] Fig. 29 is a graph of frequency component average values detected from the obtained image which is an 
Image not enlarged or reduced. The lateral axis represents the rate by which the size of the image was changed. 

5 [0225] Fig. 30 Is a graph of frequency component average values detected from the obtained Image which Is an 
Image enlarged by the rate of 1 .2. The lateral axis represents the rate by which the size of the image was changed. 
[0226] Fig. 31 is a graph of frequency component average values detected from the obtained image which is an 
image reduced by the rate of 0.8. The lateral axis represents the rate by which the size of the image was changed. 
[0227] It should be noted here that an actual image has much more frequency components than the image taken in 

10 the above example. Therefore, the frequency component graph detected from the locations where the reference in- 
formation signal are embedded becomes more complex as shown in Fig. 24. 

[0228] Specifically, this is due to the fact that the original frequency component of the image appears as the back- 
ground value. Therefore, as has been described, the influence of the background value has to be considered. 
[0229] Another mode of embodiment of the present invention is a recording medium storing a program for executing 

15 an original Image defonming-method detecting method, the method comprising: a first step of obtaining a subject image 
which looks similar to an image from a known source but whose digital watermark infomiation is not legible; a second 
step of Fourier Uanbrumiiiiy liie Subject Image into a first Fourier cpacc imago; £ third step of embeddinn a predeter- 
mined reference Infonnation signal S to a predetemnined location of the first Fourier-space image; a fourth step of 
reverse-Fourier-transfonning the Fourier-space image provided with the reference information signal into an image- 

20 space image; a fifth step of individually changing the size of the image-space image using each of enlarging/reducing 
rates selected from a predetermined range of the enlarging/reducing rates; a six step of Fourier transfonning each of 
the differently-sized Image-space images Into second Fourier-space images; a seventh step of detecting a frequency 
component values of the embedded reference information signal from each of the Fourier-space images; an eighth 
step of an-aying each of the detected frequency components sequentially according to the size of the enlarging/reducing 

25 rates; and a ninth step of detennining whether or not there Is a frequency component value outstanding from surround- 
ing frequency component values, and detennining, If there is a frequency component value outstanding from surround- 
ing frequency component values, that the obtained Image was made by changing the size of the original image at the 
identified rate. 

[0230] Further, still another mode of embodiment according to the present invention Is a recording medium storing 
30 a program for executing an original image size detecting method, the method comprising: a first step of obtaining a 
subject image which looks similar to an image from a known source but whose digital watermaric infomiation is not 
legible; a second step of Fourier transfonning the subject Image Into a first Fourier-space image; a third step of em- 
bedding a predetennined reference informatiori signal S to a predetermined location of the first Fourier-space image; 
a fourth step of reverse- Fourier-transfo mi in g the Fourier-space image provided with the reference infonnation signal 
35 into an image-space image; a fifth step of individually changing the size of the Image-space image using each of 
enlarging/reducing rates selected from a predetennined range of the enlarging/reducing rates; a six step of Fourier 
transfomnlng each of the differently-sized image-space images into second Fourier-space images; a seventh step of 
detecting a frequency component values of the embedded reference infonnation signal from each of the Fourier-space 
images; an eighth step of arraying each of the detected frequency components sequentially according to the size of 
40 the enlarging/reducing rates; and a ninth step of determining whether or not there is a frequency component value 
outstanding from surrounding frequency component values, and detennining. If there is a frequency component value 
outstanding from surrounding frequency component values, that the identified enlarging/reducing rate corresponding 
to the outstanding frequency component value is the rate at whk;h the original image was enlarged/reduced to make 
the obtained Image. 

45 [0231] Still another mode of embodiment of the present invention is a recording medium storing a program for exe- 
cuting an original-image digital-watennark detecting method, the method comprising: a first step of obtaining a subject 
image which looks similar to an Image from a known source but whose digital watermark infonnation is not legible; a 
second step of Fourier transfonning the subject image into a first Fourier-space image; a third step of embedding a 
predetermined reference infonnation signal S to a predetermined location of the first Fourier-space image; a fourth 

50 step of reverse-Fourier-transfonning the Fourier-space image provided with the reference information signal into an 
image-space image; a fifth step of individually changing the size of the Image-space image using each of enlarging/ 
reducing rates selected from a predetennined range of the eniarging/reducing rates; a six step of Fourier transfonning 
each of the differently-sized image-space images into second Fourier-space images; a seventh step of detecting a 
frequency component values of the embedded reference Infonnation signal from each of the Fourier-space images; 

55 an eighth step of an-aying each of the detected frequency components sequentially according to the size of the enlarg- 
ing/reducing rates; and a ninth step of determining whether or not there is a frequency component value outstanding 
from sun^ounding frequency component values, and detennining, if there is a frequency component value outstanding 
from sun^ounding frequency component values, that the identified enlarging/reducing rate corresponding to the out- 
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standing frequency component value Is the rate at which the original image was enlarged/reduced to make the obtained 
image; a tenth step of reproducing the original Image by applying the reciprocal number of the enlarging/reducing rate 
identified in the ninth step; and an eleventh step of searching the reproduced original Image for the watemiark infor- 
mation. 

5 [0232] Since the digital watemnark infomiatlon detecting device and the digital watemnark information detecting meth- 
od according to the present invention utilizes the technical arrangement as has been described above, the present 
Invention provides a digital watermark information detecting device and a digital watermark Information detecting meth- 
od necessary for protecting proprietary right to the Image information from Illegal infringement, by making possible to 
detect the original digital watermark infomriation from an illegally prepared still image infonnation distributed In the 

10 market, created by executing the rotating deforming operation to the original Image information which Is a still Image 
Including a predetermined digital watemiark Infonnation. 

[0233] Further, since the original-image-size detecting device and the digital -watermari< detecting method according 
to the present invention utilizes the technical arrangement as has been described above, the present invention has 
made possible to provide a device and a method enabling protection of proprietary right to the Image infonnation from 

IS illegal infringement, by making possible to detect the original digital watemiark infonnation from an illegally prepared 
still image information distributed in the mart<et, created by executing the rotating defonning operation to the original 
imdgt; iru'umiauon Wuicn is a Still imaga ir.cludlrg a prcdctcrrr^lnod digits! watermark !nform«tinn. 
[0234] Each feature disclosed in this specification (which temn includes the claims) and/or shown In the drawings 
may be incorporated in the Invention independently of other disclosed and/or illustrated features. 

20 [0235] Statements in this specification of the "objects of the Invention" relate to prefen-ed embodiments of the Inven- 
tion, but not necessarily to ail embodiments of the invention falling within the claims. 
[0236] The description of the invention with reference to the drawings is by way of example only. 
[0237] The text of the abstract filed herewith is repeated here as part of the specification. The present invention 
provides a method for detecting digital watennark Infonnation Inserted in original image information from Image infor- 

25 matlon in whbh the digital watennaric infonnation is deleted, altered or defonned by executing defonning operation to 
the original image infonnation. The digital watermaric infonnation detecting device 100 comprises: means 2 which 
calculates a matrix and a transposed matrix about a system matrix from an obtained Image infonnation; means 3 which 
calculates an estimated initial image vector from the image infonnation; means 4 which calculates a residual vector of 
the image infonnation; means 5 which calculates a square error of the residual vector; means 6 which detennines 

30 whether or not the residual vector square en^or value is a minimum value; means 7 which calculates a correcting vector; 
means 8 which calculates a reverse-rotated image; means 9 whteh substitutes an output from the reverse-rotated 
image catoulating means with a value calculated by the estimated Image vector calculating means; means 10 whbh 
obtains an estimated value of the original Image infonnation when the minimum value of the residual vector square 
error Is detected; and digital watermark Infomiatlon displaying means 11 which retrieves digital watemnaric Infonnation 

35 from the estimated Image infonnation of the original image infonnation and whbh displays the retrieved digital water- 
maric infonnation. 



Claims 

40 

1 . A digital watermark infonnation detecting device comprising: analyzing means which analyzes an obtained subject 
Image infonnation expected to contain obtained digital watenmark information, therein; Image processing informa- 
tion detennining means which determines whether the subject Image infonnation is image infonnation subjected 
to a rotating defonning operation or image infonnation subjected to an enlarging/reducing defonming operation; 

45 characteristic value calculating means which calculates a value of a predetennined characteristic to the subject 

image Infonnation based on a result of the above analysis and detennination; image Infonnation estimating means 
which estimates Image infonnation of the subject image infonnation by analyzing the characteristic value; and 
digital watennark infonnation retrieving means which retrieves digital watennark Information contained in the es- 
timated image Infonnation. 

50 

2. A digital watermark Infonnation detecting device, comprising: image infonnation analyzing means for an obtained 
Image; matrix calculating means which calculates a matrix and a transposed matrix about a system matrix, from 
the Image information; estimated initial image vector cabulating means which calculates an estimated Initial Image 
vector from the image infonnation; residual vector calculating means which calculates a residual vector between 

55 the image vector in the Image information and a value obtained by multiplying the estimated initial image vector 

with the matrix; residual vector square error calculating means which calculates a square error of the residual 
vector; square error value determining means which detennines whether or not the residual vector square error 
value is a minimum value; connecting vector calculating means which calculates a connecting vector; reverse-rotated 
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image calculating means which calculates a reverse- rotated image; adding means which adds an output from the 
reverse-rotated image calculating means to a value calculated by the estimated image vector calculating means; 
means which obtains an estimated value of the original image information from the minimum value of the residual 
vector square error when the square error value determining means detects the minimum value of the residual 
5 vector square en^or; digital watermark infomiation displaying means which retrieves digital watennark information 

from the estimated Image information about the original image information and which displays the retrieved digital 
watennark infomiation. 

3. The digital watenmark infomiation detecting device according to Claim 2, wherein the image information is stiil- 
10 image infomiation. 

4. The digital watennark Infomiation detecting device according to any one of Claims 1 through 3, wherein the image 
infomnation is presumed to have been subjected to a rotating defonning operation. 

15 5. The digital watennark information detecting devbe according to any one of Claims 1 through 4, wherein said image 
infonnation to be inspected is substantially identical or similar to the original Image infonnation, and source infor- 

rnatjnn tharonf hoinn Confirmed. 

6. The digital watennark infonnation detecting device according to any one of Claims 1 through 5, wherein the square 
20 error value detennlning means is arranged to calculate the minimum square error value by applying an affine 

inverse matrix or numerical solution by means of Jacobian method. 

7. A digital watennark information detecting method comprising the steps of: obtaining a subject image Information 
expected to contain digital watennark infonnation; performing analysis on the subject Image infonnation, thereby 

25 analyzing and detemninlng whether the subject image Infonnation is image information subjected to a rotating 

defonning operation or image information subjected to an enlarging/reducing defonning operation; obtaining a 
value of a predetermined characteristic to the subject Image information based on a result of the above analysis 
and detenni nation; estimating image infomiation of the subject image infonnation by analyzing the characteristic 
value; and retrieving digital watermark infonnation contained in the estimated image information. 

30 

8. A digital watermark infonnation detecting method, comprising the steps of: obtaining image infonnation; determin- 
ing whether or not the obtained image Infonnation was subjected to a rotating defonning operation; calculating a 
matrix and a transposed matrix about a system matrix from the image infonnation defomried by rotation; calculating 
an estimated initial image vector from the image infonnation; calculating a residual vector between the image 

35 vector in the image infonnation and a value obtained by multiplying the estimated initial Image vector with the 

matrix; calculating a square error of the residual vector; detennlning whether or not the residual vector square 
error value Is a minimum value; cateulating a correcting vector and calculating a reverse-rotated Image when the 
square error value of the residual vector is determined not to be the minimum value; adding the calculated reverse- 
rotated image infonnation to a value calculated by the estimated image vector calculating means; repeating the 

40 residual vector square error calculation as necessary; estimating original Image information from the minimum 

square error value when the square en-or value of the residual vector is detennined to be the minimum value; and 
retrieving digital watennark information from the estimated original image information. 

9. The digital watennark information detecting method according to Claim 8, wherein the image Information is still- 
45 image information. 

10. The digital watennark information detecting method according to Claim 8 or 9, wherein the image infonnation is 
presumed to have been subjected to a rotating defonning operation. 

so 11. The digital watennark infonnation detecting method according to any one of Claims 8 through 10, wherein said 
image infonnation to be Inspected, is substantially identical or similar to the original image infonnation, and source 
infonnation thereof being confimned. 

12. The digital watermark information detecting method according to any one of Claims 8 through 11, wherein the 
55 square error value detemiinlng means is arranged to calculate the minimum square error value by applying an 

affine inverse matrix or numerical solution by means of Jacobian method. 

1 3. A recording medium storing a program for executing a digital watermark infonnation detecting method by computer, 
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the method comprising: a first step of obtaining image infonnation; a second step of determining whether or not 
the obtained image infomnation was subjected to a rotating deforming operation; a third step of calculating a matrix 
and a transposed matrix about a system matrix from the image Infonnation defonned by rotation; a fourth step of 
calcuiating an estimated Initial Image vector from the image Infonnation; a fifth step of calculating a residual vector 

5 between the Image vector in the image infomrtation and a value obtained by multiplying the estimated initial image 

vector with the matrix; a sixth step of calculating a square en-or of the residual vector; a seventh step of determining 
whether or not the residual vector square error value is a minimum value; an eighth step of calculating a con-ecting 
vector when the square error value of the residual vector Is detennlned not to be the minimum value; a ninth step 
of calculating a reverse-rotated image from the con-ectlng vector; a tenth step of adding the reverse-rotated Image 

10 information thus calculated to a value calculated by the estimated image vector calculating means; an eleventh 
step of repeating the residual vector square en-or calculation as necessary; a twelfth step of estimating original 
image infonnation from the minimum square en-or value when the square error value of the residual vector is 
detennined to be the minimum value; and a step of retrieving digital watermark Infonnatton from the estimated 
original Image infonnation. 

15 

14. An original image defomning-method detecting method, comprising the steps of: embedding a predetermined ref- 
erence infonnation signal to a predetermined location of a Fourier-space image prepared by Fourier transfomrilna 
an obtained subject image; reverse-Fourier-transforming the Fourier-space image into an image-space image; 
individually changing the Image size of the image-space Image by using each oneof a plurality of enlarging/reducing 

20 rates selected from a predetermined range of the enlarging/reducing rates; Fourier transforming again each of the 

differentiy-sized image-space images into Fourier-space images; detecting frequency components of the embed- 
ded reference infonnation signal previously embedded Into the image infonnation from each of the Fourier-space 
images; arranging the frequency components of a plurality of the reference infonnation signal detected from the 
a second Fourier-space Images of a plurality of the image-space Images each having a different size from the 

25 other, sequentially according to the size of the enlarging/reducing rates; and detennlning that the subject image 

was made by enlarging/reducing a prescribed original Image when it can be detennined that there is a frequency 
component value at a certain enlarging/reducing rate, outstanding from surrounding frequency component values. 

15. The original Image deforming-method detecting method according to Claim 14, wherein the subject image is an 
30 image substantially the same as or close to the original image whose source Infonnation Is Icnown. 

16. The original image defomning-method detecting method according to Claim 14 or 15, wherein the original image 
and the obtained image are still images. 

35 17. The original image defomning-method detecting method according to any one of Claims 14 through 16, wherein 
the predetennined range of the enlarging/reducing rate is from 0.5 through 2.0. 

18. The original image defonning-method detecting method according to any one of Claims 14 or 17, wherein a back- 
ground value is removed from all of the detected frequency components when detennlnation about whether or not 

^ the frequency component value is outstanding from the sun-ounding frequency component values, Is made. 

19. An original Image size detecting method comprising the steps of: embedding a predetennined reference infonnation 
signal to a predetennined location of a Fourier-space Image prepared by Fourier transforming an obtained image; 
reverse-Fourier-transforming the Fourier-space Image Into an Image-space Image; individually changing the image 

45 size of the image-space image using each one of a plurality of enlarging/reducing rates selected from a predeter- 

mined range of the enlarging/reducing rates; Fourier transfonning again each of the differently-sized Image-space 
Images Into Fourier-space images; detecting frequency component values of the embedded reference infonnation 
signal previously embedded Into the image to be inspected, from each of the Fourier-space Images; arranging the 
frequency components of said reference infonnation signal detected from the Fourier-space images of the image- 

50 space images each having a different size from the other, sequentially according to the size of the enlarging/ 

reducing rates; and detennlning, when there is a frequency component value at a certain enlarging/reducing rate 
outstanding from surrounding frequency component values, that the obtained image was made by enlarging/re- 
ducing an obtained original image at the enlarging/reducing rate corresponding to a frequency component value 
which is outstanding from surrounding frequency component value. 

55 

20. The original image size detecting method according to Claim 1 9, wherein the subject image and the original Image 
are images whose source is known. 
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21. The original image size detecting method according to Claim 19 or 20, wherein the obtained subject image is an 
image generally the same as or similar to the original image. 

22. The original image size detecting method according to any one of Claims 1 9 through 21 , wherein the original image 
5 and the obtained subject image are still images. 

23. The original image size detecting method according to any one of Claims 1 9 through 22, wherein the predetemnined 
range of the enlarging/reducing rate is from 0.5 through 2.0. 

10 24. The original image size detecting method according to any one of Claims 19 through 23, wherein a baclcground 
value is removed from all of the detected frequency components when determining whether or not there Is a a 
peak value of a frequency component value. 

25. A digital-watermark detecting method comprising the steps of: reproducing an original image by applying to an 
IS obtained subject image a reciprocal number of the enlarging/reducing rate at which the size of the original image 

was altered to that of the subject image, the rate being obtained by the original image size detecting method 
according to Claims 19 through 24: and detecting the reproduced original image for a watennark. 

26. An original image deform ing-method detecting device, for detecting the size of the original image from an obtained 
20 image, said device comprising: a first Fourier transfomriing means which Fourier transfonns the obtained image 

into a first Fourier-space image; reference infonnation signal embedding means which embeds a predetermined 
reference Infonnation signal at a predetemnined location of the Fourier-space image prepared by Fourier trans- 
forming the obtained image; reverse Fourier transfomriing means which reverse- Fourier-transfomns the Fourier- 
space image into which the reference infonnation signal being imbedded into an image-space Image ; enlarging/ 

25 reducing rate storing means which stores a plurality of enlarging/reducing rates for changing the size of the image- 

space Image; image-size changing means which sequentially selects desired enlarging/reducing rates from the 
enlarging/reducing rate storing means, and sequentially and individually changes the Image size of the image- 
space image by individually using each of the selected enlarging/reducing rates; second Fourier transforming 
means which Fourier transfonns again each of the image-space images the image signal of which being changed 

30 into second Fourier-space images; frequency component value detecting means which detects a frequency com- 

ponent value of the reference infonnation signal from each of the second Fourier-space Images; frequency com- 
ponent value arranging means which arrays a plurality of frequency component values obtained from each of the 
second Fourier-space images into a predetermined sequence according to the enlarging/reducing rates; outstand- 
ing frequency component value detemriining means which detennines whether or not there is a plurality of frequency 

35 component values sequentially an-anged; and original-image enlarging/reducing-defonnation identifying means 

which detemnlnes, on the basis of an output value outputted from the outstanding frequency component value 
detennining means, whether or not the subject image is a result of enlarging/reducing defonning operation per- 
fonned to the original image or not. 

<o 27. The original image deforming-method detecting device according to Claim 26, wherein the outstanding frequency 
component value detennining means has noise removing means which removes a background value from the 
frequency component values. 

28. An original-image-size detecting device, for detecting the size of the original image from an obtained image, the 
45 device comprising: first Fouriertransforming means which Fouriertransforms the obtained image into a first Fourier- 

space image; reference infonnation signal embedding means which embeds a predetermined reference informa- 
tion signal at a predetennined location of the Fourier-space Image prepared by Fouriertransforming the obtained 
image; reverse Fourier transfonning means which reverse-Fourier-transfonns the Fourier-space image into which 
the reference infonnation signal had been embedded therein, Into an image-space image ; enlarging/reducing rate 
50 storing means which stores a plurality of enlarging/reducing rates for changing the size of the image-space image; 

image-size changing means which sequentially selects desired enlarging/reducing rates from the enlarging/reduc- 
ing rate storing means, and sequentially and Individually changes the image size of the image-space image by 
individually using each of a plurality of the selected enlarging/reducing rates; second Fouriertransforming means 
which Fourier again transforms each of the image-space images the image size of which being changed, respec- 
55 tively, into second Fourier-space images; frequency component value detecting means which detects a frequency 

component value of the reference information signal from each of the second Fourier-space Images; frequency 
component value arranging means which arrays a plurality of frequency component values obtained from each of 
the second Fourier-space images Into a predetennined sequence according to the enlai^ing/reducing rates; out- 
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standing frequency component value detemriining means which detemnines whether or not there is a frequency 
component value outstanding from surrounding frequency component values; and original-image size identifying 
means which determines, when the outstanding frequency component is found, that the enlarging/reducing rate 
corresponding to the outstanding frequency component value is the enlarging/reducing rate whereby the image 
5 size of said obtained image had been changed from that of said original image. 

29. The original-image-size detecting device according to Claim 28, wherein the outstanding frequency component 
value detemilning means has noise removing means which removes a background value from the frequency com- 
ponent values. 

10 

30. A digital-watennark detecting device comprising the original-image-size detecting device according to claim 28 or 
29; original-image reproducing means which reproduces an original image by applying to an obtained subject 
image, a reciprocal number of the enlarging/reducing rate detemiined by the original-image-size detecting device, 
and digital watemiark information detecting means which detects whether or not the digital watermark infomnation 

IS existed in said reproduced original image by said original image reproducing means. 

31. An original image defomning-method detecting method, comprising: 

a first step of obtaining a subject image which looks similar to an original image from a known source but 
20 whose digital watemnark Information is not legible; 

a second step of Fourier transforming the obtained subject image into a first Fourier-space image; 

a third step of embedding a predetenmined reference infomnation signal to a predetemriined location of the first 

Fourier-space image; 

a fourth step of reverse-Fourier-transfonming the Fourier-space Image into which the reference infomnation 
25 signal had been embedded therein, into an image-space image; 

a fifth step of individually changing the size of the image-space image using each one of a plurality of enlarging/ 
reducing rates selected from a predetemriined range of the enlarging/reducing rates; 
a six step of Fouriertransfomriing again each of the differently-sized image-space images into second Fourier- 
space images; 

30 a seventh step of detecting af requency component value of the reference infomnation signal which is previously 

embedded in said image, from each of the Fourier-space images; 

an eighth step of arranging each of the detected frequency component values of said reference Information 

signal, sequentially according to the size of the enlarging/reducing rates; and 

a ninth step of detemnining whether or not there is a frequency component value outstanding from surrounding 
35 frequency component values at a certain enlarging/reducing rate among said sequentially arranged frequency 

component values, and determining, when there is the frequency component value outstanding from surround- 
ing frequency component values, that the obtained subject image was made by applying the enlarging/reducing 
image defonning operation to a prescribed original image. 

40 32. An original image size detecting method, comprising: 

a first step of obtaining a subject image which looks similar to an original image from a known source but 
whose digital watermark infomnation is not legible; 

a second step of Fourier transforming the obtained subject Image into a first Fourier-space image; 
45 a third step of embedding a predetermined reference information signal to a predetermined location of the first 

Fourier-space image; 

a fourth step of reverse-Fourier-transforming the Fourier-space image provided with the reference information 
signal into an image-space image; 

a fifth step of individually changing the size of the image-space image using each one of a plurality of enlarging/ 
50 reducing rates selected from a predetermined range of the enlarging/reducing rates; 

a six step of Fourier transfomning again each of the differently-sized image-space images into second Fourier- 
space images; a seventh step of detecting a frequency component value of the embedded reference informa- 
tion signal from each of the Fourier-space images; 

an eighth step of arranging each of the detected frequency components of said respective reference informa- 
55 tion signals, sequentially according to the size of the enlarging/reducing rates; and 

a ninth step of detemnining whether or not there is a frequency component value outstanding from surrounding 
frequency component values at a certain enlarging/reducing rate among said sequentially arranged frequency 
component values, and determining, when there is the frequency component value outstanding from surround- 
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ing frequency component values, that the obtained subject image was made from said original image by chang- 
ing said image size with applying the enlarging/reducing rate corresponding to said outstanding frequency 
component value, to said original image. 

5 33. The original Image size detecting method according to Claim 32, wherein the original Image is an Image whose 
source Is known. 

34. The original image size detecting method according to Claim 32 or 33, wherein the obtained image has an image 
generally similar to the original Image. 

10 

35. The original image size detecting method according to any one of Claims 32 through 34, wherein the original image 
and the obtained image are still images. 

36. The original image size detecting method according to any one of Claims 32 through 35, wherein the predetennined 
15 range of the enlarging/reducing rate Is from 0.5 through 2.0. 

37. The original image size detecting method according to any one of Claims 32 through 36, wherein a background 
vaiue is removed from all ot the detected frequency components when determining whether or not there is a 
frequency component value outstanding from the surrounding frequency component values. 

20 

38. An original image size detecting method, comprising: 

a first step of obtaining a subject image which looks similar to an image from a known source but whose digital 
watermark Information is not legible; 

a second step of Fourier transfomriing the obtained subject image into a first Fourier-space image; 
a third step of embedding a predetemiined reference infonnation signal to a predetermined location of the first 
Fourier-space image; 

a fourth step of reverse- Fourier-transfonning the Fourier-space image provided with the reference infomiation 
signal into an image-space image; 

a fifth step of Indivlduaily changing the size of the image-space image using each one of a plurality of enlarging/ 
reducing rates selected from a predetemnined range of the enlarging/reducing rates; 
a six step of Fourier transfomning again each of the differently-sized Image-space images Into second Fourier- 
space images; 

a seventh step of detecting a frequency component value of the embedded reference Information signal pre- 
viously embedded Into said Fourier-space images, from each of the Fourier-space Images; 
an eighth step of arranging each of the detected frequency components of said reference infonnation signal, 
sequentially according to the size of the enlarging/reducing rates; and 

a ninth step of detennining whether or not there is a frequency component value outstanding from surrounding 
frequency component values at a certain enlarging/reducing rate among said sequentially arranged frequency 
component values, and detennining, when there is the frequency component value outstanding from surround- 
ing frequency component values, that the obtained subject image was made from said original image by chang- 
ing said Image size with applying the enlarging/reducing rate corresponding to said outstanding frequency 
component value, to said original image. 

a tenth step of reproducing an original image by applying a reciprocal number of the enlarging/reducing rate 
obtained in the ninth step; and 

an eleventh step of detecting whether or not a watermark information existed in said original image. 

39. A recording medium storing a program for executing an original image size detecting method, the method com- 
prising: 

so 

a first step of obtaining a subject Image which looks similar to an original image from a known source but 
whose digital watemnaric infonnation is not legible; 

a second step of Fourier transforming the obtained subject image into a first Fourier-space image; 
a third step of embedding a predetemnined reference infonnation signal to a predetennlned location of the first 
55 Fourier-space image; 

a fourth step of reverse- Fourier- transform ing the Fourier-space image provided with the reference information 
signal into an image-space image; 

a fifth step of Indivlduaily changing the size of the image-space image using each one of a plurality of enlarging/ 
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reducing rates selected from a predetermined range of the enlarging/reducing rates; 
a six step of Fourier transforming again each of the differently-sized image-space images into second Fourier- 
space Images; 

a seventh step of detecting a frequency component value of the embedded reference information signal pre- 
5 viousty embedded into said Fourier-space Images, from each of the Fourier-space Images; 

an eighth step of arranging each of the detected frequency components of said reference information signal, 
sequentially according to the size of the enlarging/reducing rates; and 

a ninth stepofdetennining whether or not there is a frequency component value outstanding from surrounding 
frequency component values at a certain enlarging/reducing rate among said sequentially arranged frequency 
10 component values, and determining, when there is thefrequency component value outstanding from surround- 

ing frequency component values, that the obtained subject image was made from said original image by chang- 
ing said image size with applying the enlarging/reducing rate corresponding to said outstanding frequency 
component value, to said original image. 

IS 40. A recording medium storing a program for executing an original image size detecting method, the method com- 
prising: 

a first step of obtaining a subject image which looks similar to an original image from a Icnown source but 
whose digital watennark infomiation is not legible; 
20 a second step of Fourier transforming the subject image Into a first Fourier-space image; 

a third step of embedding a predetemiined reference infomnation signal to a predetemiined location of the first 
Fourier-space image; 

a fourth step of reverse-Fourier-transforming the Fourier-space image provided with the reference infonnation 
signal into an image-space image; 
25 a fifth step of individually changing the size of the image-space image using each one of a plurality of enlarging/ 

reducing rates selected from a predetemnined range of the enlarging/reducing rates; 
a six step of Fourier transfomiing again each of the differently-sized Image-space images Into second Fourier- 
space images; 

a seventh step of detecting a frequency component value of the embedded reference infomiation signal pre- 
50 viously embedded into said Fourier-space images, from each of the Fourier-space images; 

an eighth step of arranging each of the detected frequency components of said reference information signal, 
sequentially according to the size of the enlarging/reducing rates; and 

a ninth step of determining whether or not there is a frequency component value outstanding from surrounding 
frequency component values at a certain enlarging/reducing rate among said sequentially arranged frequency 
35 component values, and detemaining, when there is the frequency component value outstanding from surround- 

ing frequency component values, that the obtained subject image was made from said original image by chang- 
Ing said image size with applying the enlarging/reducing rate corresponding to said outstanding frequency 
component value, to said original image. 

a tenth step of reproducing an original image by applying a reciprocal number of the enlarging/reducing rate 
40 obtained in the ninth step; and 

an eleventh step of determining the original image for watermark infomnation. 
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(c) FIFTIETH ESTIMATION VALUE 
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(d)ONE HUNDREDTH ESTIMATION VALUE 
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SIGNAL 



FOURIER TRANSFORMATION S'9''AL FERQUENCY 



(a: IMAGE SPACE) 




(b: FOURIER SPACE) 
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r-, ENERGY-DIFFUSED WATERMARK 



ORIGINAL SIGNAL 




INVERSE FOURIER 
TRANSFORMATION 



(c: IMAGE SPACE). 



(d: FOURIER SPACE) 



(A) 
2 
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(a ; PER 1 0D=8P i xe I s. LENGTH=256P i xe I s) 




(b ; 1 28P i xe I sSamp I e ; Rate=1 . 0) 



-20-16-12-8 -4 0 4 8 12 16 20 
(c;FREQUENCY=1 28/8=1 6) 



ONE-DIMENSIONAL TIME SERIES HAVING A PERIOD OF 8p(p=pixela) 

AND A LENGTH OF 256p IS SAMPLED BY THE SIZE OF THE ORIGINAL IMAGE. 
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(A) 
2 



F i g. 20 

(a : PER 1 0D=8P i xe I s. LENGTH=256P i xe I s) 




(b : 1 28P i xe ! sSamp I e : Rate=1 28/1 60 «0 , 8) 



(c:FRE0UENCY>16) 



ONE-DIMENSIONAL TIME SERIES HAVING A PERIOD OF 8p(p=pixels) 
AND A LENGTH OF 256p IS REDUCED AND THEN SAMPLED. 
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(a ; PER 1 0D=8P i xe I s. LENGTH=256P i xe I s) 




107p-128p 
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(b ; 1 28P 1 xe I sSamp I e : Rate= 1 28/1 07 w 1 . 2) 



(c ; FREQUENCY < 16) ' 

ONE-DIMENSIONAL TIME SERIES HAVING A PERIOD OF 8p(p=pixeis) 
AND A LENGTH OF 256p IS ENLARGED AND THEN SAMPLED. 



43 



EP 1 156 660 A2 



F i g. 2 2 



REDUCED PIXEL 
(R=0.8. f>16) 




ORIGINAL IMAGE 
(R=1.0.f=16) 



10-r 
9" 



ENLARGED PIXELS" ENLARGED PIXEL 
(R=1.2.f<16) 7-- (R=1.2.f<16) 

6 -- 
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4-- 
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1 + 



ORIGINAL IMAGE 
(R=1.0.f=16) 





REDUCED PIXEL 
(R=0. 8.f>16) 



;^T^ 6*t^*^2 14 16 18 ^ 



IF TIME SERIES SIGNALS ARE ENLARGED/REDUCED. THEIR 
FREQUENCY COMPONENTS BECOME SMALL/LARGE. 
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ZERO-FREOUENCYCOMPONENT OF IMAGE ZERO-FREOUENCYCOMPONENT OF IMAGE 



^ ORIGINAL LOCATION .o 
" ^ LXATIOH AFTER MOVE 



[WHEN AN IMAGE IS REDUCED.I 
LOCATION OF THE EMBEDDED I 
WATERMARK SIGNAL WILL I 
aiOVE TOWARD THE CENTER J 



(a) FOURIER TRANSFORMATION VALUE 
OF A REDUCED IMAGE 



LOCATION AFTER MOVE,' 
o ^ORIGINAL LOCATION «' 

iwHBi an" mage Ts"enLarged] 

ILOCATION OF THE EMBEDDED i 
IWATERMARK SIGNAL VIILL i 
!J¥)VEJiWAY_FRpM THE^CENTCRj 



(b) FOURIER TRANSFORMATION VALUE 
OF AN ENLARGED IMAGE 



AN IMAGE WAS REDUCED/ENLARGED. AND THEN SAMPLED BY THE SIZE 
OF 128X128PIXELS. FOURIER TRANSFORMED VALUES OF THE SAMPLED 
IMAGES ARE SHOWN IN (a) AND (b) RESPECTIVELY. 
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0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1:4 1.5 1.6 1.7 1.8 1.9. 10 
ENLARGING/REDUCING RATE. 



AVERAGE VALUE OF FREQUENCY COMPONENTS FROM ALL WATERMARK • 
EMBEDMENT LOCATIONS AT EACH ENURGiNG/REDUCING RATE 
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END 
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OBTAIN SUBJECT IMAGE 




EMBED REFERENCE INFORMATION 
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REVERSE DFT 



ST-IOl 
-ST-2 

'ST-4 



ENLARGING/REDUCING OPERATIONn ^ST-5 



[ 



DFT AGAIN 



DETECT FREQUENCY COMPONENT 



ARRAY AND INTEGRATE FREQUENCY 
COMPONENT VALUES 




DETERMINE ENLARGE/REDUCE 



'ST-6 
'ST-7 
v_--ST-8 
'ST-9 



'ST-90 



IMAGE FROM A DIFFERENT 
SOURCE 
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END ) 



[DETERMINE A ENLARDING/REDUCING RATE b ^$T-91 

REPRODUCE ORIGINAL IMAGE BY ENLARGING/h^ ST-1 0 
REDUCING SUBJECT IMAGE BY RATE 1/D 




^ TAMPERED IMAGE FROM THE SME SOURCE ST-1 4 
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IVI IMAGE OF A 128x128 
l2SJ BLOCK-SIZE 



^56/1. 2=21 3pixelg ^^ (I?) (a) 
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128pixels 



• ORIGINAL IMAGE 
> REDUCED IMAGE 
^ ENLARGED IMAGE 
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T) ,=L/L,=0. 5 
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